AI Infrastructure: From Pilot Projects to Commercial Deployment

How Businesses Are Changing Their Approaches to Implementing Artificial Intelligence, and Why Infrastructure Readiness Is Becoming a Key Success Factor.

Today, the adoption of artificial intelligence in the corporate sector faces a paradoxical situation: the quality of models is no longer the main limitation. The main barriers have shifted to the area of infrastructure readiness—information security requirements, cost control, scalability, and data handling are coming to the forefront. How are business needs evolving, and which approaches to AI implementation are becoming industry standards?

From Abstract Experiments to Applied Tasks

Over the past year, there have been fundamental changes in what companies are asking for. Whereas businesses used to come with vague requests to “try out AI” or to follow management directives, the situation today is fundamentally different. Clients now come with clearly defined business objectives: to reduce first-line support response times, speed up analysts’ work, improve search capabilities within the internal knowledge base, or automate specific workflows.

The most important trend is a conscious approach to project economics. Companies demand transparency: they need to understand how much computing resources each department consumes, which subscriptions are in use, and what the additional costs per employee workstation are. Artificial intelligence is ceasing to be an end in itself and is becoming a practical tool with clear economics and measurable results.

The economic factor is also shifting interest away from large-scale, universal models toward more localized scenarios. A year ago, many were eager to roll out something grand, but the high cost of such projects led to a rethinking of approaches. Today, the trend is toward locally deployed models tailored to specific departments and tasks: marketing, first-line support, searching the internal knowledge base, and automating straightforward business processes.

Where to Start for a Successful Implementation

A key stage of any AI project is defining the business objective and establishing performance metrics. Without this, there is a high risk of building a solution that will go unused. Experts recommend starting with a detailed description of the existing process: how many people are involved, how much time the work takes, where errors occur, and what their cost is. The most important question is: what will change if the process is accelerated—say, by a factor of two?

This is where a serious problem lies: employees know their routine inside and out, but often don’t understand how it can be optimized using AI. They lack the time or specialized knowledge to determine where a RAG pipeline is needed, where it makes sense to incorporate speech recognition, and where AI won’t have any tangible impact at all. Therefore, the company must have a person or team that views processes holistically and understands where automation will truly be beneficial.

This is followed by architectural design, where a key decision arises: does the company need full control over its data, or is it willing to adopt a hybrid approach? If data cannot be transferred outside the company’s perimeter, this calls for an on-premises infrastructure with models deployed locally within a secure environment. In this case, the company gains full control over its data, protection of trade secrets, and the ability to manage the infrastructure independently.

When It’s Better to Postpone an AI Project

There are clear red flags that indicate it’s premature to start an AI project. The most obvious one is the lack of digitized processes. If a company doesn’t understand exactly what it wants to improve - but simply “wants AI” - the project is doomed to fail. By default, artificial intelligence does not fix chaos—it rather scales what already exists.

The second critical factor is the lack of a change owner within the company. There must be a person responsible for the results, the budget, and the transformation process itself. Without an understanding of the cost and expected benefits, the project risks turning into an endless experiment.

An important caveat: on-premises neural networks, especially in large-scale deployments, require significant investment. Many companies, focusing on the subscription costs of external services, mistakenly assume that their own infrastructure will cost roughly the same. In reality, for large businesses and large-scale implementations, in-house infrastructure can end up being many times more expensive. However, this approach offers greater control - the company does not risk having commercially sensitive data uploaded by employees to external services.

A Platform-Based Approach as a Solution to Infrastructure Challenges

Specialized platforms are being developed specifically to manage AI infrastructure, allowing companies to configure scenarios with control, auditing, and data transfer management. Without such a solution, a company essentially just grants employees access to external services, losing track of what data is leaving the organization and which scenarios are permissible.

Hardware-software complexes (HSCs) for AI tasks are primarily geared toward the corporate sector with high security and transparency requirements: finance, manufacturing, and the public sector. These are industries where data must not leave the company’s perimeter, and cloud APIs and external services are often simply not considered.

Such solutions include a library of pre-built models - updated, maintained, and tested for various scenarios. The customer does not have to start from scratch but can select models suited to their tasks and deploy them within a secure environment.

A second important scenario involves serving different departments with different models, permissions, and quotas. Companies need to see who is consuming how many resources and what the cost is. Dashboards allow the IT director or business leader to track infrastructure usage and its monetary cost.

The third scenario involves low-code workflow builders. Not every company has a strong development team, yet process automation is essential. For example, to speed up the processing of support requests, you can use real-time streaming transcription or call post-processing, transferring data to a CRM or ticketing system. Such workflows are built not through full-scale development from scratch, but in a more application-oriented manner.

A Builder, Not a One-Size-Fits-All Solution

A software-as-a-service (SaaS) platform for AI tasks is more of a builder than a one-size-fits-all template. Every company has different needs, so the infrastructure is tailored to the specific workload: servers, GPUs, and the required performance. The required models are deployed, the necessary services are connected, and on top of that, a sequence of actions is built according to the company’s needs.

If the customer understands the number of users, the expected volume of requests, and the tasks to be solved, both the architecture and the hardware can be precisely selected. This is not a “one-size-fits-all” approach, but rather a configuration tailored to a specific workload and specific business processes.

The traditional approach involves the company independently selecting hardware, testing models, and building solutions. In the initial phase, this is a perfectly viable approach: almost everyone starts with a pilot project, run on existing infrastructure or a rented virtual machine with a GPU. But the transition from the pilot to production turns out to be the most challenging stage.

As long as only a few people are using the solution, much of the work can be done manually. When it comes to department-wide adoption, entirely different challenges arise: hardware and software support, MLOps or technical support roles, scaling, integration with corporate accounts and access roles, auditing, monitoring, data preparation, and anonymization. In a pilot, some things can be done quickly and even manually. In a production environment, that no longer works. An AI toolkit provides many features “out of the box.” Implementing these on your own could take half a year or a year and undermine trust in the tool itself.

Hardware and Cooling

In most scenarios- about 90% of cases- four-unit servers based on the NVIDIA MGX architecture are used, allowing for the installation of up to eight GPUs, including top-of-the-line NVIDIA H200 accelerators. The key factor is the cooling system. Eight 600-watt GPUs generate nearly 5 kilowatts of heat, which must be effectively dissipated. There are solutions on the market that look similar but are designed for cards with significantly lower power consumption. Such details are often buried in the fine print, and it later turns out that the server isn’t designed to handle the required load.

For scenarios involving gradual scaling from pilot projects to industrial deployment, there are eight-unit servers equipped with desktop-class graphics cards such as the RTX 4090 or RTX 5090. Major brands like Dell or Hewlett Packard offer virtually no such solutions. This approach is in demand not only for AI but also for high-performance computing, mathematical modeling, medical research, and drug development - anywhere GPU-based computations are needed but a highly fault-tolerant “24×7” infrastructure isn’t strictly required. If the task isn’t mission-critical and computations are run periodically, using desktop GPUs can significantly reduce the total cost of ownership without compromising results.

The Economics of AI Infrastructure

The main capital expense is the GPU-equipped servers themselves. But people often forget that simply buying a server isn’t enough - it needs to be housed somewhere. GPU infrastructure requires a completely different level of power consumption compared to conventional servers for virtualization or corporate systems. Power and cooling requirements are increasing. If a company houses its equipment in a data center, hosting costs also rise. In Russia, this is not yet always viewed as a critical expense, but this is precisely where limitations often arise. Not every data center today is ready to host such capacity - some lack space, others lack sufficient power, or face cooling constraints.

This is particularly noticeable when companies attempt to deploy infrastructure on their own premises, such as at manufacturing sites. It often turns out that the existing server room is not designed to handle either the required power capacity or the necessary cooling. In such cases, the project begins to require additional investment in the data center’s engineering infrastructure.

That said, there is a positive aspect to AI: a great many tools are available as open source and do not require additional licensing. In this sense, the economics often turn out to be simpler than in traditional enterprise applications.

Common Mistakes in Cost Planning

The first mistake is to consider only the cost of the GPU server and fail to account for infrastructure limitations. It’s important to understand in advance where the equipment will be located. Not every data center is ready to accommodate GPU servers, and not every server room can handle the power and cooling demands. Sometimes it turns out that implementing AI requires a complete overhaul of the infrastructure, which entails entirely different costs.

The second issue relates to cloud models. If a company uses external services, it’s important to understand right away how costs will be calculated and what restrictions need to be put in place. The problem is that AI implementation usually happens gradually. At the start, there are few users, and the bill seems quite manageable. But then employees begin to realize how much it speeds up their work; usage increases, and along with it, costs begin to rise rapidly.

Billing becomes a critical issue. You need to see who is using how much, which departments are consuming resources, and what it’s costing the company. Modern platforms include such mechanisms - you can track the use of on-premises infrastructure and external services and view the entire cost structure in a single view. This helps avoid a situation where expenses start to rise imperceptibly and spiral out of control.

The GPU Market and the Search for Alternatives

Today, NVIDIA holds a certain monopoly, especially in the segment of training large language models. Essentially, this segment remains in NVIDIA’s hands for now. At the same time, the market is already beginning to look for alternatives - not because everyone wants to abandon NVIDIA at any cost, but rather because businesses want to reduce their dependence on NVIDIA and lower the cost of their solutions.

There is growing interest in Chinese manufacturers and specialized GPU solutions. While they cannot yet fully compete in training the largest models, they already appear to be a perfectly viable option for specific application scenarios. This involves a more specialized approach. It’s not necessary to build a massive infrastructure around a universal model that has to do everything at once. Increasingly, companies want to solve a specific task - speech analytics, document search, or image processing—and select more specialized hardware for it.

This is one of the key trends for the coming years. Whereas the market previously moved toward the largest and most general-purpose models possible, there is now a gradually emerging demand for specialization - where a specific architecture and a specific type of accelerator are selected for a specific task. There’s a sense that by 2028, we’ll see far more solutions based on Chinese GPUs and less dependence on NVIDIA, especially in the segment of applied enterprise tasks.

A Look into the Future

The cost of implementing AI should decrease significantly, and the market will most likely move in this direction. Gigantic general-purpose models will gradually give way to a set of small, specialized models. Companies will no longer have a single system that “can do everything,” but rather a set of AI agents tailored to specific roles and tasks.

Artificial intelligence will become a tool not only for developers or people involved in coding. It will also be useful for regular office employees and frontline staff. This will also affect infrastructure: instead of massive GPU clusters, the market will seek more affordable and specialized solutions. Customers increasingly want infrastructure tailored to specific business scenarios rather than a universal “large model.” This means that dependence on NVIDIA and other American manufacturers may gradually decrease. We will likely see many more solutions based on Chinese and possibly other Asian technologies.