By Jamila REJEB, Data Scientist at LittleBigCode

Both DevOps and MLOps aim to place a piece of software in a repeatable and fault-tolerant workflow, while MLOps adds a machine learning component to that software.

However, MLOps ultimately has a few more nuances for each component of the workflow that differ from traditional DevOps such as data labelling, data transformation, feature engineering, and algorithm selection process.

Source : MLOps vs DevOps : What is the difference ? PHDATA.IO (Click on image for link)

DataOps is a collaborative data management practice focused on improving communication, integration, and automation of data flows between data businesses (Data Engineers) and data consumers (Data Scientists, Business Analysts).

The promise of DataOps is to improve and optimize the life cycle of Data & Analytics projects in terms of speed and quality. Like DevOps, DataOps refocuses your projects on collaboration.

Why do we need MLOps ?

MLOps motto: optimize, accelerate and secure an ML project lifecycle.

This set of best practices will help you with versioning: code, data, models… testing and validation in case of a data drift, or to establish the value of the solution (A/B testing), and ensure the stability of the solution through unit tests for example. It will help us also with monitoring and alerting by understanding models using interpretability and visualizing live metrics. Moreover, MLOps is widely used for automation and orchestration purposes thanks to CI/CD pipelines and DAG orchestration. We could even add things like data and privacy security and budget control.

As a matter of fact, within our data-science team, MLOps tools and practices help us in every step of a Machine Learning project. Starting with logging all the experiments in one comprehensive dashboard makes it easier to compare and choose the best model for a set of metrics that we define to automate its deployment to production and ensure the required tests are in place.
But what if we need to version this model or the data on which it was trained? MLOps can also help with that. In addition, we have been able to easily keep track of the model performances and be alerted in case of a drift.
In a nutshell, using this set of practices help us better organize our ML project and communicate more easily.

Since a picture is word thousand words, here is a graph in attempt to better summarize the concepts addressed above.

In a more practical way, MLOps will help you answer the following questions :

• How can I be notified as soon as possible whenever my data distributions change?
  Things evolve over time, your data too. To name but one example, if you are classifying text using a supervised model, and you start receiving data in a new language unless you detect this change in your data distribution, you will not be able to take this new need into consideration and update your model accordingly.

• How can I track my experiments?
  An ML project lifecycle involves multiple phases of experimentations and iterations like testing multiple models, multiple feature selections, hyperparameter tuning… Storing results in Excel sheet plain files will rapidly turn out to be tedious. Fortunately, numerous solutions exist today to track and visualize these experiments.

• How can I easily deploy my model in production?
  Once your ML project is approved to be used by the business you need to choose, how are the operations going to use its predictions? A common solution would be to expose these predictions using an API. Some solutions that we will discuss in the next section aim to ease the deployment and shorten the delays.

• How can I make sure that my models are still accurate?
  A deterioration in model performances can undermine consumer trust and introduce risk to the business. Therefore, we need to continuously monitor the model behavior and test its predictions.
  The faster you detect an eventual problem, the quicker you can remediate it.

• How can I manage the lifecycle of all production models?
  Models are bound to change over time. We need to be able to easily retrain a model and update it in production or even plan an automatic retrain with minimal effort. We should also be able to easily manage multiple model versions in production.

• How can we explain the predictions?
  In some cases, the operations need to explain the model’s predictions. Depending on which model is being used a range of tools can help clarify its behavior in order to enhance trust in AI.

What tool can be used ?

Numerous methods have been developed in an attempt to encompass everything described above. As a matter of fact, when conducting the search there were much too many to include, so we decided to list the most common ones. Every tool has its functionalities. We attempted to obtain a general benchmark of what each tool accomplishes. We will dive into a more thorough comparison in the next article.

Conclusion

MLOps is undoubtedly a novel concept, but one that is well justified. It provides organizations with a clear framework for more effective deploying and monitoring machine learning projects to bring business interest back to the forefront of your ML operations. Using these sets of tools and best practices, it becomes easier for data scientists, data engineers, software engineers, and the operations to work and communicate efficiently.