Costs of AI/ML: How Expensive Is it to Develop and Maintain AI Solutions?
In the past few months, we have been closely monitoring emerging economic trends and the government’s fiscal and monetary policy decisions to better understand their impact on the AI space. Recently released Autumn Statement 2022 and the Economic and fiscal outlook – November 2022 lay down rather bleak growth projections for the next few quarters. We expect the AI industry to feel the impact of several rapidly shifting economic variables in the oncoming months.
In recent years AI has permeated all aspects of businesses, from automating manual tasks to developing high-level business insights. The contribution of AI to organisational processes has become so significant that companies may feel compelled to adopt AI to keep up with the competition.
While adopting AI-based solutions can bring massive boosts to the productivity of business workflows, excessive costs, now reverberated by declining revenues and rising energy prices, can prevent companies from adopting and scaling AI. While there is no easy price estimation, the costs of developing, testing, and deploying a basic AI system can reach up to $50,000. The costs of developing and deploying an AI solution can largely be attributed to hardware, software, and data. Figure 1 below gives a finer breakdown.
Figure 01: Breakdown of costs in delivering AI, originally published in How to Decrease the Cost of AI – Krista
In the current economic climate, investing in a costly AI solution may have to be thought over twice. For those who have already implemented AI solutions, the costs of maintaining and updating these systems can be a significant pain point. This is particularly the case for those who may be currently facing rising costs in cloud-based deployment or energy consumption.
While these costs vary with the number of predictions made and the size of the models trained, the expenditure of maintaining an AI solution also can change with other factors that often go unnoticed. Mitigating some of these factors can make AI solutions far less expensive, especially shielding companies against cost increases that happen during times of an economic downturn. In this article, we look at the costs of bad code, financially and otherwise, and explore approaches to mitigating such costs.
The Unexpected: Costs of Bad Code Run Deeper than You Think
Driven by the need to experiment more with data science elements of machine learning model development, those in the data science pipeline may spend less time refining code to achieve optimal code quality. Read more on this in our article, How Can Data Scientists Write Production-quality Machine Learning Code? Bad code adds to the costs of maintaining AI solutions, and its burden often goes unnoticed.
A quick search will reveal that bad code wastes developer time, takes up resources, and, ultimately, reduces business profitability. There are many estimations of the monetary value of losses caused due to bad code. For instance, according to Stripe calculations, bad code equates to nearly $85 billion worldwide in opportunity costs lost annually. Even though the value of investing in maintaining code quality is often highlighted, the problem of sub-par code seems to exacerbate, with no definitive solutions.
Figure 02: Impact of code quality on developers, originally published in https://stripe.com/files/reports/the-developer-coefficient.pdf
The Stripe poll takes input from developers to understand their pain points, and we see that maintaining legacy systems is at the top of the list of factors that hinder developer productivity (Figure 02). We also see that factors related to bad code bases (insufficient time to fix poor quality code, spending too much time on legacy systems, and paying down technical debt) contribute most to dampening developer morale. Since commercial development environments are under immense time pressure to maintain a competitive advantage, maintaining developer morale can bring about a significant boost to the AI applications deployment pipeline. Data Analysts and Scientists, AI and Machine Learning Specialists, and Software and Application Developers are among the top 10 job roles that will see increasing demand between 2020-2025, which emphasises the need to create development spaces that are productive and less strenuous on employees.
Most importantly, it is necessary to find ways to improve code quality and the implementation of machine learning models sustainably.
How Can evoML Help with AI Cost-saving?
AI applications are subject to change along three axes: model, code, and data. We look at two of these axes to understand how TurinTech’s evoML platform can optimise at the model and code level for cost-efficient AI. We further illustrate our evoML-related cost savings using an organisational scenario.
- Model-level Optimisation
ML models are often at the core of AI applications. Optimising the process of producing ML-based predictions is one way to reduce the costs of implementing AI solutions.
ML models can be implemented using different frameworks/libraries, where some implementations can be more cost-efficient than others.
However, converting an ML model to a different format is complex. It requires developers with knowledge of the frameworks, who will have to conduct a manual conversion to the desired implementation.
A solution to this is utilising intermediate representations (IR) – a type of middleman. IRs extract the original format and structure of the model and convert it into the desired format, allowing developers to circumvent the manual conversion process.
Figure 03: Role of Intermediate Representations (IR) in model optimisation, originally published in A friendly introduction to machine learning compilers and optimizers
evoML provides cutting-edge functionalities to convert existing ML models to different implementations using comparable libraries. By converting ML model code to more efficient frameworks/libraries, companies can save significantly when deploying AI pipelines.
Figure 04 shows the cost of 10M inferences of an ML model, using different implementations executed by evoML. The best implementations, ONNX and TVM, are almost 98% less costly to implement. When deployed in a commercial environment these cost savings convert to millions of dollars.
Figure 04: Cost savings from model-level optimisation, developed based on original TurinTech data
With TurinTech’s signature optimisation technology we looked at the runtime improvements that can be made to an NLP model during implementation. evoML took an original model and boosted the model within a matter of seconds to be implemented in ONNX, PyTorch, and TVM. Solutions generated this way include tuning hyperparameters such as batch size.
These are the improvements we noted:
Figure 05: Boosts to the runtime of models with different implementations, based on TurinTech data
Implementing the model on ONNX showed a boost of 262.35%, which translates to massive savings in terms of resource costs and efficiency.
- Code-level Optimisation
Another unforeseen cost of deploying AI and ML comes from the execution of inefficient code. Code that is inefficient consumes more memory, energy, and time to execute, leading to a higher cost of resources. Code optimisation is an often under-utilised solution to achieving high-quality and efficient code bases. Since optimising code can often be cumbersome, it can easily be sidestepped.
However, evoML and its automated code optimisation functionalities can easily identify the most expensive lines of code that affect the speed of a model during training and prediction and optimise these lines of code. This can significantly bring down the costs of executing ML code, and thereby entire AI applications.
Recent TurinTech research found that code optimisation could improve the execution time of specific ML codebases by up to around 20%. When the optimised code was evaluated in an Azure-based cloud environment, cost savings of around 30% per hour were observed, for the virtual machine size used.
Cost Savings and Value Addition from evoML
We use the following hypothetical scenario to break down evoML-related value gains. Consider a company that has 10 full-time employees using evoML, who are recruited at an annual salary of $100,000 per employee. The company has 4 ML projects in production for the year, with an average annual business value per ML project at $500,000.
During cloud-based deployment of evoML, model and code optimisation will bring about significant value gains for companies in terms of productivity, accuracy, efficiency, and sustainability of the ML pipeline. Our estimates show that with evoML, companies can make cost-savings close to a massive $2 million a year for ML model production and deployment while allowing businesses to meet their sustainability goals faster.
A breakdown of these costs is in Figure 06 below:
Figure 06: Breakdown of evoML-related cost savings, based on TurinTech ROI calculations
While economic forecasts might look bleak for the next few months, your expectations for AI adoption should not be. With a carefully developed strategy that successfully utilises cutting-edge optimisation technologies, we are certain that you will be able to progress with your AI adoption, implementation, and maintenance plans at a steady pace.
1 Stripe partnered with Harris Poll to survey developers, technical leaders and C-level executives about their organizations’ business challenges, software development practices, and future investments to determine the role that developer productivity plays in their success—and the growth of worldwide GDP overall. More than 1,000 developers and more than 1,000 C-level executives in the United States, U.K., France, Germany, and Singapore participated in the study.