Token counter and context window calculator

Why token counts matter more than word counts

A language model does not read words. It reads tokens — sub-word fragments produced by a byte-pair encoding — and every limit that constrains your work is expressed in them: the context window, the price per call, the truncation point where your carefully assembled sources silently fall off the end.

Character counts and word counts are poor proxies. Code, JSON, URLs, non-English text and long identifiers all tokenize far more expensively than prose, sometimes at three times the rate. If you are budgeting a retrieval pipeline or debugging why a prompt got cut off, an estimate that is 30% wrong is not useful.

What this tool shows you

Paste or drop text and you get, immediately and locally:

• The token count, plus characters, words and the characters-per-token ratio your text actually achieves.
• Estimated input cost across a set of common model prices, so a long prompt can be compared before you send it.
• A context-window fill bar per model — the fastest way to see that a 180k-token dossier will not fit where you were about to put it.

It tells you when it is guessing

The count comes from o200k_base, the byte-pair encoding GPT-4o and GPT-4.1 use. It is loaded from this site the first time you run the tool, and the count is exact for those models. Claude, Llama and Gemini each use their own tokenizer, so their true counts differ somewhat; treat the figure as a close guide for them.

If the tokenizer cannot be fetched — you are offline, or opening the page from a file — the tool falls back to a heuristic and says on screen that the number is an estimate.

That labelling is deliberate. A silently wrong token count is worse than an admitted approximation, because it is the kind of error you only discover after a pipeline has been built on top of it.

Where it fits in a research workflow

Token budgeting is the first constraint you hit when you stop chatting and start building context. It decides how many chunks you can retrieve, how much of a source you can quote, and whether a long report can be produced in one pass or has to be assembled.

FlexContext exists for that problem at the other end of the scale: keeping a growing library of documents, pages, transcripts and datasets, and choosing precisely which parts of it a model sees for a given question.