Why Is AI Inference Becoming the Next Enterprise Cost-Control Agenda?

The technology decision is shifting from choosing the “best AI model” to orchestrating the right level of intelligence for each task. Gartner argues that more capable agentic systems can increase total inference expenditure because sophisticated, multistep workflows require substantially more inference than simple interactions. EY reaches a similar conclusion at workflow level: frontier models should be used selectively for hard-reasoning steps, while lower-cost models can handle routine tasks where less capability is sufficient. Forrester adds a structural alternative: smaller fine-tuned models, often built from open-weight models, are becoming economically practical for more specialized workloads.

The economic rationale for multimodel architectures is becoming measurable. Accenture reports that 54% of AI requests are routed to models more capable and expensive than the task requires, while its analysis of more than 9,000 tasks found that approximately 90% did not require frontier-model capability. EY identifies additional consumption drivers including expanding context, repeated retrieval, tool calls, retries and agent loops. Its recommendation is to measure consumption by workflow and task before attempting platform-level optimization.

AI consumption decisions are increasingly distributed across developers, employees and business teams. Developers select models. In AI-assisted work, longer sessions and expanding context can increase consumption. Agentic workflows introduce additional consumption through chained actions and repeated processing. Each choice can alter inference cost.

Accenture therefore frames tokenomics as a cross-functional management discipline involving CIOs, CFOs, CHROs and business leaders. EY shows why: ordinary AI-assisted work patterns can become repeated model consumption even though they were never designed as economic decisions. PwC extends the workforce question further by arguing that redesigned AI workflows require new roles, skills and leadership responsibilities.

Visibility remains weak. Only 53% of token usage can currently be traced to a particular user or team in Accenture's research, while only 7% of surveyed companies use chargeback for AI costs. This matters because teams behave differently when consumption has an accountable owner. EY recommends training engineers to recognize seven recurring consumption patterns and manage AI efficiency at workflow level rather than focusing only on model price. PwC adds that its workflow design and simulation framework can identify likely role changes, skill needs and organizational impacts before the new workflow is built.

Traditional enterprise technology spending followed relatively predictable economics, with investments in software, infrastructure and talent expected to deliver value over several years. AI behaves differently. Every model call, agent step and inference event can carry incremental cost.

KPMG argues that this moves AI investment from a conventional technology-budget decision toward continuous economic management. BCG organizes the response into three levers: user behavior, infrastructure controls and commercial sourcing. The implication is larger than cost control. AI changes how enterprises budget technology, establish financial accountability and source AI capacity.

The hidden cost base is material. KPMG reports that 49% of organizations in its cited AI Pulse survey have delayed or reduced AI deployments where benefits did not clearly outweigh costs. It also estimates that original investment cases may represent only 25–35% of total three-year cost of ownership, as integration, retraining, talent, security and monitoring costs can be dispersed across budgets and functions. BCG similarly warns that token consumption is growing faster than token prices are declining.

The most important financial shift is moving from cost per token to cost per completed work. McKinsey & Company shows why. In one banking customer-service example, token consumption represents only 20–25% of variable run cost, while human oversight represents 70–75%. Optimizing model price alone can therefore miss the larger economic opportunity: reducing exception rates and simplifying review processes that require costly human intervention. Accenture reaches the same conclusion from another direction: fewer than one in five dollars of token spend can currently be linked to quantified financial outcomes.

The ROI opportunity can be substantial when the workflow is selected correctly. In an illustrative banking onboarding example, McKinsey & Company estimates the total cost to complete onboarding could decline from approximately $50–$150 per customer to $10–$30, while the completed process still requires several agents, deterministic systems and human oversight teams. Accenture finds only 35% of companies can calculate cost per business outcome even for their largest AI use case. The measurement gap is therefore becoming a material constraint on AI ROI assessment.

Inference economics will not converge on a single enterprise model. Sector requirements differ. Deloitte finds that financial services and energy organizations place comparatively greater emphasis on regulatory compliance, while consumer and financial-services respondents highlight GPU availability and operating costs. Technology, media and telecommunications organizations place greater weight on keeping insight generation in-house and responding rapidly to competition.

This suggests that the optimal AI economic model will vary by industry: cost efficiency may dominate some workloads, while latency, sovereignty, risk management or regulatory requirements can justify more expensive AI capabilities elsewhere.

The projected consumption curve is significant. Among technology, media and telecommunications respondents surveyed by Deloitte, the share expecting to consume more than 10 billion tokens per month rises from 29% to 71% by 2028. In financial services, the equivalent projection rises from 36% to 70%. Across the five industries studied, respondents identify innovation capacity at 71%, risk management at 64%, and token optimization at 59% as leading expected outcomes from AI factories. These are projections rather than realized results, but they indicate where infrastructure and governance planning is moving.