What Are Tokens in AI Models? Costs and Limits Explained

If you use an AI model through an API, almost everything is measured in tokens: the price you pay, the amount of text the model can read at once, how long its answer can be and how fast it responds. Understanding tokens is the quickest way to make AI apps cheaper and more reliable.

This guide explains what tokens are, how to estimate them and how to use that knowledge to control costs.

What is a token?

A token is a chunk of text that a language model treats as a single unit. It is not exactly a word or a character — it is something in between, chosen by an algorithm called a tokenizer.

Common words are usually one token. Longer or rarer words get split into several pieces. Spaces and punctuation are often attached to the following word. For example, a tokenizer might split a sentence like this:

"Tokenization isn't hard."
→ ["Token", "ization", " isn", "'t", " hard", "."]

The exact split depends on the model: each model family uses its own tokenizer, so the same text can produce different token counts in different models.

Why not just use words or characters?

• Characters would make sequences very long, which makes models slower and more expensive to run.
• Whole words would need an enormous vocabulary and could not handle typos, new words or code.
• Subword tokens are the compromise: a vocabulary of tens of thousands to a few hundred thousand pieces that can spell anything.

Rules of thumb for estimating tokens

For English text with most modern tokenizers:

Measure	Approximate tokens
1 token	~4 characters, or ~¾ of a word
100 words	~130–140 tokens
1 page (~500 words)	~650–700 tokens
1,500-word article	~2,000 tokens

These are estimates. Some content uses noticeably more tokens:

• Code: symbols, indentation and variable names split into many small tokens.
• Non-English languages: many languages, especially those not using the Latin alphabet, need more tokens for the same meaning.
• Numbers, URLs and IDs: random strings like UUIDs or hashes tokenize poorly.
• JSON: quotes, braces and repeated keys add up quickly.

For a quick estimate and a cost calculation, paste your text into our token and cost estimator. For exact counts, use the token-counting tool or endpoint your provider offers, or their published tokenizer library.

Why tokens matter

1. Cost

AI APIs charge per token, usually quoted per million tokens, with separate prices for:

• Input tokens: everything you send — system prompt, conversation history, documents, tool definitions.
• Output tokens: everything the model writes back. Output is typically several times more expensive than input.

A simple cost formula:

cost = (input_tokens / 1,000,000 × input_price) + (output_tokens / 1,000,000 × output_price)

Example with made-up prices of $3 per million input tokens and $15 per million output tokens: a request with 2,000 input tokens and 500 output tokens costs

(2,000 / 1,000,000 × $3) + (500 / 1,000,000 × $15) = $0.006 + $0.0075 = $0.0135

That looks tiny, but at 100,000 requests a month it is $1,350. Check your provider's current pricing page — prices change often and vary a lot between models.

2. The context window

The context window is the maximum number of tokens a model can consider in one request — input and output combined. If your prompt plus the expected answer exceeds it, the request fails or old content has to be dropped.

Modern models have context windows ranging from tens of thousands to over a million tokens. A large window does not mean you should fill it: more input means higher cost, slower responses, and sometimes the model pays less attention to details buried in the middle.

3. Output limits

Separate from the context window, most APIs have a maximum output tokens setting. If the answer hits that limit, it is cut off mid-sentence. When you see truncated responses, check whether the API reported a "max tokens" stop reason and raise the limit or ask for a shorter answer.

4. Speed

Models generate output one token at a time. Long answers take longer, and very long inputs increase the time before the first token appears. Asking for concise output is the easiest way to make an AI feature feel faster.

Chat history is re-sent every time

A detail that surprises many developers: APIs are usually stateless. In a chat app, every new message re-sends the whole conversation so far as input tokens.

Turn 1:  system (500) + user (100)                              → 600 input tokens
Turn 2:  system (500) + user (100) + assistant (300) + user (100) → 1,000 input tokens
Turn 3:  … + assistant (300) + user (100)                        → 1,400 input tokens

Costs grow with every turn. Long-running chats should summarize or trim old messages.

Seven ways to reduce token usage

1. Trim the system prompt. It is sent with every request. Remove repetition and examples that do not change the output.
2. Send only relevant context. Retrieve the few paragraphs that matter instead of whole documents (this is the core idea behind RAG, retrieval-augmented generation).
3. Ask for shorter output. "Answer in at most three sentences" or a strict JSON schema saves expensive output tokens.
4. Use prompt caching if your provider supports it. Repeated prefixes such as long system prompts or reference documents can be billed at a discount on later requests.
5. Pick the right model size. Smaller, cheaper models handle classification, extraction and simple rewriting well. Save the largest models for hard reasoning.
6. Batch non-urgent work. Many providers offer discounted batch processing for jobs that can wait.
7. Measure. Log the input and output token counts the API returns with each response, and look at which features consume the most.

Estimating tokens in code

A quick character-based estimate is fine for budgeting and for checking whether something will fit before you send it:

def estimate_tokens(text: str) -> int:
    """Rough estimate for English text: about 4 characters per token."""
    return max(1, len(text) // 4)

prompt = open("article.md", encoding="utf-8").read()
print(estimate_tokens(prompt))

For billing-accurate numbers, rely on the usage figures the API returns in every response — they are what you are charged for.

FAQ

Is a token the same as a word?

No. In English, one word averages about 1.3 tokens. Short common words are one token; long or unusual words are several.

Do spaces and punctuation count as tokens?

They are part of tokens. A space is usually merged into the next word's token, and punctuation is often its own token.

Why did my request fail with a context length error?

Your input plus the requested maximum output exceeded the model's context window. Shorten the input, trim conversation history or lower the maximum output setting.

Do images count as tokens?

For models that accept images, yes — images are converted into tokens too, and larger images generally cost more. Providers document how they calculate this.

Further reading

• Your provider's documentation on token counting, pricing and prompt caching
• Related: How to write better prompts for AI coding assistants
• Related: SQLite vs PostgreSQL for side projects