Box CEO says companies must budget for rising AI token costs

Box CEO Aaron Levie said companies need to plan for their AI compute budgets, warning that token consumption won’t be limited to engineers. He wrote that while AI use will "of course start in engineering," it will "eventually hit the rest of knowledge work as well," citing legal and sales as potential big token users.

Tokens determine how AI is measured and priced: they are units of text, a word or part of a word, and the building blocks of large language models that power chatbots. They are not a flat fee, and AI companies tend to charge more for using the most advanced models and for asking more complicated questions.

Levie’s post responded to Nvidia CEO Jensen Huang, who said he would be "deeply alarmed" if an engineer paid $500,000 didn't use tokens equivalent to at least half that salary. Huang added, "I'm going to ask them how much did you spend in tokens?

box, aaron levie, ai tokens, token costs, ai budgets, nvidia, jensen huang, llms, knowledge work, token pricing