Measuring AI's True Value: CFOs Unveil New Metrics to Ensure Economic Gains

As businesses continue to pour billions of dollars into artificial intelligence (AI) research and development, a growing concern is whether these investments are yielding tangible economic returns. According to OpenAI's Chief Financial Officer, Sarah Friar, CFOs and business leaders must move beyond simplistic metrics like cost per token and adopt a more nuanced approach to evaluating AI's effectiveness. In a recent blog post, Friar revealed the scorecard she uses to determine whether AI investments are paying off, and her insights are set to revolutionize the way companies measure AI's true value.

Background & Context

For years, software success was measured through adoption metrics, such as seats, active users, and renewals. However, AI requires a more sophisticated approach, one that assesses the actual work it accomplishes. As AI becomes increasingly integral to business operations, companies are grappling with the challenge of quantifying its economic impact.

With the global AI market projected to reach $190 billion by 2025, CFOs are under pressure to demonstrate the financial benefits of AI investments. Moreover, the rapidly evolving AI landscape demands that companies stay ahead of the curve, adapting their strategies to optimize returns on investment.

Key Details

In her blog post, Friar emphasized that the fundamental question facing CFOs and business leaders is whether the value of the work AI completes grows faster than the cost of producing it. To answer this question, she advocates for a more in-depth analysis, focusing on the "useful intelligence per dollar" metric. This involves evaluating four critical elements: the quality of the work AI completes, the cost of each successful task, the reliability of the results, and the increase in value as usage grows.

Practically speaking, this means tracking the volume of AI-completed work that meets a defined quality bar, calculating the full cost of completing that work, and then dividing by the number of successful tasks to obtain a cost per successful task. The ultimate test is whether people can reliably depend on the output and whether, over time, high-quality completed work grows faster than total cost while quality holds or improves.

Friar's approach underscores the importance of compute in driving AI's economic value. As a private company, OpenAI does not publish formal capex guidance, but its Stargate initiative has already made significant strides in building large-scale AI infrastructure in the U.S. With a 10-gigawatt capacity goal by 2029, OpenAI is poised to further accelerate AI adoption and demand.

What Experts Say

The rising expectations for CFOs to determine strategy, including AI spend, have been reflected in recent events. The McKinsey Global CFO Forum, a premier gathering of finance chiefs from around the world, has highlighted the critical role CFOs play in driving business decisions. As Andy West, a senior partner at McKinsey, noted, "CFOs are increasingly expected to help determine strategy, including where the company places its biggest long-term bets, like AI spend, alongside the CEO."

Friar's emphasis on useful intelligence per dollar resonates with experts in the field, who stress the need for a more nuanced approach to evaluating AI's economic impact. By focusing on the quality of work completed, cost per successful task, reliability of results, and increase in value as usage grows, companies can make informed decisions about AI investments and ensure a higher return on investment.

Key Takeaways

• Measuring AI's true value requires a more sophisticated approach, focusing on the quality of work completed, cost per successful task, reliability of results, and increase in value as usage grows.
• The "useful intelligence per dollar" metric is a critical framework for evaluating AI's economic impact, emphasizing the importance of compute in driving AI's value.
• CFOs are increasingly expected to determine strategy, including AI spend, and drive business decisions, underscoring the critical role they play in driving company growth.
• Companies must adapt their strategies to optimize returns on investment, staying ahead of the curve in the rapidly evolving AI landscape.

What This Means For You

As AI becomes increasingly integral to business operations, companies must adopt a more nuanced approach to evaluating its economic impact. By focusing on the quality of work completed, cost per successful task, reliability of results, and increase in value as usage grows, companies can make informed decisions about AI investments and ensure a higher return on investment. This shift in perspective will enable businesses to harness the full potential of AI, driving growth and competitiveness in an ever-evolving market.

As you consider your company's AI strategy, remember that measuring AI's true value requires a more sophisticated approach. By embracing the "useful intelligence per dollar" metric and focusing on the quality of work completed, cost per successful task, reliability of results, and increase in value as usage grows, you can ensure that your AI investments are yielding tangible economic returns and driving business growth.