Source: Government Technology | Author: Abby Sourwine | Publication date: 28 Aug 2026
AI models measure prompts, uploaded materials and output generation in terms of “tokens,” often with an associated cost beyond user licenses. Experts say these costs can be difficult to predict and budget for.
For universities adopting generative artificial intelligence, the cost of the technology is not always captured by the price of a software license. Many AI models can incur charges based on token usage, a way of breaking down inputs and measuring their compute costs.
However, tokens are not a widely understood currency, and failure to appropriately estimate token usage can lead to unexpected costs, said Chris Kapusta, vice president of advisory and transformation at IT consulting group GDT.
“Tokens are kind of the hidden cost of AI today,” he said.
According to OpenAI, tokens are the units that AI models use to process and generate information. A user’s prompt is broken into tokens before a model processes it, and the model’s response is generated using tokens as well. Many AI providers charge separately for input and output tokens, typically calculating the price per million tokens.
That means two people can use the same AI platform but generate very different costs. A short question-and-answer may consume relatively few tokens, while a long prompt containing a research paper or large document can consume many more.
For users of the University of Michigan’s own Maizey AI model, for example, a one-sentence prompt followed by a paragraph response costs less than 1 cent, and a 75-word prompt followed by a one-page reply costs 1 to 2 cents, according to the university website. This kind of token cost breakdown can represent chatbots answering quick student questions about housing or orientation, said Joseph Kim, CEO of Druid AI.
For other university functions, like research, costs go up. For the Maizey model, a prompt and response involving 45 pages of text can cost as much as 88 cents.
While the individual amounts are small, the challenge for universities lies in predicting how many such interactions will occur, how large they will be and which models will handle them, Kapusta said.
Why tokens are hard to budget for
Traditional software is often easier to budget for because an institution can pay a relatively predictable amount for a specific number of users or licenses. Kapusta said many AI tools have this per-user cost, but they also have an associated usage allowance.
For example, the University of Connecticut announced in June that its agreement with OpenAI would change from a set-price ChatGPT Edu subscription with open access to basic and advanced models to a subscription-plus-usage model. Beginning July 1, 2026, users would continue to have access to basic features with their subscription, while advanced models would incur an additional charge for credits used.
These usage costs can be obscured by language like “tokens” and “credits.” For example, ChatGPT’s agreement with the University of California, Davis, offers users 100 credits per week.
According to OpenAI’s website, ChatGPT credit usage is determined by the following formula: total credits = (input tokens / 1,000,000 × input rate) + (cached input tokens / 1,000,000 × cached input rate) + (output tokens / 1,000,000 × output rate).
Kapusta said financial operations have not yet caught up to this form of measurement, and financial leaders don’t have reliable prediction models for token usage across a university’s many different functions.
“The admissions department in a higher education organization who are using AI to filter through applications for prospective students are going to have a very different usage profile than, say, the finance department, who are using it to help manage and accelerate loan approvals and financial aid approvals,” he said.
In addition to variation among users, there is also variation in usage costs across different models. Kim said text and voice interactions can have very different token profiles, for example. One second of voice interaction can involve 100 to 200 tokens, he said, even for simple prompts.
In research examining token consumption in agentic coding tasks, researchers found that the agentic tasks they studied consumed roughly 1,000 times more tokens than coding and task chats. The research also found that token consumption can vary substantially even when an agent performed the same task — costs varied by as much as 30 times in the study.
Kim said Druid AI has chosen not to license its AI products based on token usage because it is so complicated.
“We don’t license it this way on purpose,” he said, “because it’s such a complex thing.”
What institutions can do
When budgeting for AI tools, Kapusta said the many different uses across university workflows should not be treated as a single category in an institution’s AI budget. Instead, schools should compile a list of use cases they want their AI tools to be used for — accelerating research, answering student questions — and consider what kind of AI model would be best suited for the job. For advanced research, perhaps a more powerful, higher-cost model is appropriate.
“[For Anthropic tools], Opus is a much heavier model usage than, say, Sonnet or Haiku,” he said. “Do I need a sledgehammer every time with Opus, or do some of the smaller ones, depending on what I’m asking, make better sense?”
Then, institutions can deploy AI systems gradually to better understand typical usage and establish a baseline for budgeting.
Kapusta said organizations are beginning to use more granular limits, allocating portions of a token pool to individual users or groups rather than allowing unrestricted access.
The Consortia Advancing Standards in Research Administration Information recommends avoiding long, flat-rate commitments because the AI market is changing quickly, and so may usage costs.
“If you’re setting an AI budget, you have to find ways to back into that budget,” Kapusta said. “There’s not great cost calculators like we have in other parts of the industry that’ll map that out for us, but if we, on the front end, scrutinize what are our heavy token use cases versus our lighter ones, that helps us determine budgeting.”
