Harnessing the Potential of MLOps
Maximizing Machine Learning Efficiency and Performance
The wonder of today’s powerful machine learning (ML) models lies in their ability to rapidly resolve challenging business problems that once took hours or days of human effort) (or were not even possible to solve.
Indeed, from finely targeted customer segmentation models and supply chain optimisation to anti-money laundering and faster development of new medicines, the application of ML has become as diverse as the number of business problems there are to solve.
“When I first worked on a segmentation model, we managed a total of five segments including young adults, seniors and so on. But with ML, there’s no limit to the number of possible segments. In fact, every customer is effectively a ‘segment of one’ and I can pitch an offer to each one – tailored perfectly for their situation.”
Solving a business problem with ML requires far more than just identifying the best algorithm and training data.
In fact, a Gartner study cited that 85% of models never actually make it into production.
This is because Machine Learning projects executed by data scientists typically involve a sequence of steps, each of which can be highly complex in its own right:
- Understanding the business problem
- Defining the business case for investment
- Identifying and accessing suitable data
- Exploring the data
- Preparing the data
- Enriching data through feature engineering
- Testing and selecting the best algorithm from a vast array of options
- Selecting the model
- Training the model
- Optimising the model
- Reviewing the model for bias, ethical and compliance aspects
- Establishing parameters for monitoring drift
- Integrating the model into production IT systems
- Monitoring ongoing performance
- Retraining or retiring/replacing the model
While the failure to adequately perform any of these steps could potentially derail an ML project, integrating the model into product IT infrastructure often proves the most challenging. That’s because models are frequently developed in environments that differ significantly from the IT department’s production ecosystem.
Furthermore, once you consider the overhead of conducting all these processes manually for every data science project, it becomes clear why so many well-intentioned and even well-designed models fail to make it into production. This challenge is often referred to as the “85% problem.”
How MLOps (Machine Learning Operations) solves the 85% problem
MLOps methodology transforms the ML paradigm by streamlining, automating and standardising many steps in the ML productization process. According to McKinsey, the approach enhances productivity by over 30%, empowering data science teams to churn out successful models more efficiently, with less effort and reduced friction. By upgrading and automating manual, non-scalable, repeatable tasks, such as infrastructure management, it makes data scientists more efficient and productive, enabling them to focus on their core business – and thereby removing many obstacles that create the ‘85% problem’.
In the table below, see how traditional, manual steps for developing and implementing a model compares to performing these steps in an MLOps-enabled environment.
|
Without MLOps |
With MLOps |
|
Manually obtain data from best available sources |
Access data from data marketplace |
|
Copy data into computable storage |
Provision new sandpit on cloud infrastructure |
|
Local/manual data engineering. Code not version controlled and probably not retained for future reuse |
Access enterprise data engineering toolsets – with CI/CD pipelines, version control, traceability, auditability, and reusability |
|
Local/manual feature engineering |
Feature engineering using enterprise toolsets |
|
Install/reuse favourite algorithms |
Access algorithms from enterprise repository |
|
Choose winning algorithm based on prediction accuracy |
Select winning algorithm |
|
Reach out to relevant parties for peer review, ethics review, etc. |
Submit model to automated review process |
|
Work with IT to integrate the model into production systems and workflows, each time using a bespoke approach |
Promote model to production |
|
No version control, no branch control (for iterative experiments), no documentation (we’re too busy for that) |
Version and branch control available for integrating future experiments/variations on the same model |
|
Ad-hoc note-taking documentation when free time available |
Documented as part of CI/CD process |
|
Lost knowledge of process and learnings |
Knowledge of process and learnings retained |
|
Process is not repeatable and so, is not auditable |
Process is repeatable and auditable |
|
Ad-hoc monitoring of model in production |
Automated monitoring in production |
|
Without MLOps |
With MLOps |
|
Manually obtain data from best available sources |
Access data from data marketplace |
|
Copy data into computable storage |
Provision new sandpit on cloud infrastructure |
|
Local/manual data engineering. Code not version controlled and probably not retained for future reuse |
Access enterprise data engineering toolsets – with CI/CD pipelines, version control, traceability, auditability, and reusability |
|
Local/manual feature engineering |
Feature engineering using enterprise toolsets |
|
Install/reuse favourite algorithms |
Access algorithms from enterprise repository |
|
Choose winning algorithm based on prediction accuracy |
Select winning algorithm |
|
Reach out to relevant parties for peer review, ethics review, etc. |
Submit model to automated review process |
|
Work with IT to integrate the model into production systems and workflows, each time using a bespoke approach |
Promote model to production |
|
No version control, no branch control (for iterative experiments), no documentation (we’re too busy for that) |
Version and branch control available for integrating future experiments/variations on the same model |
|
Ad-hoc note-taking documentation when free time available |
Documented as part of CI/CD process |
|
Lost knowledge of process and learnings |
Knowledge of process and learnings retained |
|
Process is not repeatable and so, is not auditable |
Process is repeatable and auditable |
|
Ad-hoc monitoring of model in production |
Automated monitoring in production |
Getting started with MLOps
MLOps isn’t a technology that you can buy off the shelf and implement. Rather, it’s an integrated combination of organizational processes, roles and skillsets that need to be orchestrated to support those processes, alongside the technologies that support the teams and processes. Implementing MLOps involves organizational process mapping and organizational change, as well as the implementation of technologies to support model and code management and deployment.
A strategic approach and roadmap are therefore required.
Building and deploying MLOps strategy and use cases
At Amdocs , our mature and proven approach empowers you to implement a robust and scalable MLOps practice. This includes working across your organization to ensure all stakeholders are engaged and empowered to embrace the target state business architecture, as well as the new technologies and ways of working.
Across the globe, we’ve assisted banks and other regulated enterprises achieve high maturity and productivity, leveraging our robust and flexible framework tailored for an MLOps strategy and roadmap. For example, we recently designed and deployed an MLOps strategy for Singapore’s leading digital bank – not just enabling technologies, but also business processes, supporting roles, organizational structure and change management. We’ve also deployed high-value and high-impact ML use cases on customer platforms, including:
- Hyper-personalisation
- Anti-money laundering
- Fraud detection
- Pricing optimisation
- Next best experience
- Credit scoring
For more information on the Amdocs MLOps offering, click here
