What is a Token in AI?

Imagine you're teaching an alien who has never seen human language before. How would you start? You'd probably begin with the building blocks—individual letters, then combine them into words and sentences.

That's exactly how AI learns to understand language.

Token: The "Lego Brick" of AI

A token is the smallest unit of text that an AI model reads and processes. Think of it as a Lego brick—the basic piece from which all language is built.

Here's the key insight: a token is not the same as a word.

A token can be:

• A whole word (like "AI" or "learning")
• A piece of a word (like "en" + "code" + "ing")
• A punctuation mark (like "." or "!")
• A space
• Even a single character

Let's look at some examples

English example:

"AI is transforming our world"

Might be broken into: ["AI", " is", "transform", "ing", " our", " world"]

Chinese example:

"我爱学习人工智能"

Might be broken into: ["我", "爱", "学习", "人工智能"]

Notice how "transforming" got split into three pieces? That's typical—AI keeps common words intact but breaks longer or more complex words into smaller parts.

Why does this matter?

If you've used ChatGPT or similar AI tools, you might have noticed:

1. Memory limits — AI can only handle around 4,000 to 100,000 tokens at a time (depending on the model). Think of it like AI's short-term memory—beyond that, it "forgets."

2. Response length — If your input takes up most of the token limit, the AI has less room to answer.

A practical rule of thumb

OpenAI offers this useful guide: about 4 characters ≈ 1 token, or roughly 75 English words ≈ 100 tokens.

This is less accurate for Chinese, since each Chinese character typically counts as one token.

---

Coding Plan vs Token Plan: From Unlimited Subscription to Pay-Per-Use

If you've used AI services from cloud providers like Tencent Cloud or Alibaba Cloud, you might have noticed: they used to call it "Coding Plan," but now it's all "Token Plan."

It's not just a name change—it's a complete shift in the business model.

What is Coding Plan?

Coding Plan was a low-cost subscription service offered by major cloud providers in 2025-early 2026.

Take Tencent Cloud as an example. Their old Coding Plan Lite cost just 40 RMB ($5-6) per month, and you could call the AI coding service as many times as you wanted—no limits, no caps, just unlimited usage within the validity period.

This model had several characteristics:

• Fixed monthly fee: Pay a set amount each month, regardless of actual usage
• Unlimited calls: Within the time window, call as many times as you want
• Simple: No need to worry about tokens per request

Using AI coding back then was like going to an all-you-can-eat buffet—pay once, and seafood, steak, desserts, all you can eat.

The Pros of Coding Plan

1. Reassuring for users — One-time payment, no shock bills at month-end
2. Carefree usage — Use as much as you want without watching the meter
3. Lower barrier to entry — Low price let more developers try AI programming

The Cons of Coding Plan

But this model had a fatal flaw: it didn't account for actual token consumption.

A simple Q&A might only need a few hundred tokens. But an AI coding agent handling a full code review could consume hundreds of thousands—even millions—of tokens—1,000x more.

This led to an absurdity:

• Light users: Ask a few questions per month, getting a "deal"
• Heavy users (especially Agent users): One task consumes 100x more tokens than a simple Q&A

Cloud providers realized: the more they sold, the more they lost.

Why Switch to Token Plan?

In early 2026, three factors forced cloud providers to change their pricing model:

1. AI Agents consume way more tokens than expected

Take OpenClaw (the popular AI coding agent at the time) as an example—one Agent task could consume 10x to 100x more tokens than a simple Q&A. When users ran lots of Agents, the fixed Coding Plan fee simply couldn't cover costs.

2. Severe supply-demand imbalance

• Daily token usage in China exceeded 140 trillion by early 2026, up 40% from late 2025
• But GPU supply couldn't keep up: Nvidia's export restrictions, limited TSMC capacity, domestic server price hikes
• Result: Compute costs rose, providers couldn't bear it

3. Commercial pressure

The era of burning cash for users is over. Providers need to profit. Selling low-priced Coding Plans meant providers lost money on heavy users—had to shift to finer-grained billing.

So starting March 2026, cloud providers began phasing out Coding Plan and moving to Token Plan:

What Changed with Token Plan?

The shift completely changed the landscape:

• Higher entry barrier — Used to be $5-6/month unlimited; now starts at $27+ per month, plus pay-per-use
• More token-sensitive — Every call shows exactly how many tokens consumed, cost is transparent
• Heavy users might get a better deal — If you use a lot, pay-per-use could be cheaper than fixed monthly (depends on task type)

In short: Coding Plan = "unlimited monthly subscription", Token Plan = "pay-per-use". The former suits stable regular usage; the latter suits variable complex tasks.

Now you understand why cloud providers are phasing out Coding Plan—compute is just too expensive to sustain the old model.

How Much Do AI Tokens Cost?

AI providers charge for token usage in two main ways: pay-as-you-go (per token) and subscription (monthly fee for a quota of tokens).

Pay-as-you-go pricing is straightforward: you pay a fixed rate per token. For example, if a provider charges $0.01 per 1,000 tokens and your application uses 500,000 tokens, you'd pay $5. This model is best for variable workloads.

Subscription plans offer a set number of tokens or credits for a monthly fee. MiniMax, Xiaomi MiMo, and Zhipu GLM offer tiered subscription plans ranging from $10 to $160 per month. These are ideal for teams with predictable usage.

The cheapest provider for API tokens is currently DeepSeek V4, while Cursor offers the most advanced AI-native coding experience at $20/month. Use TokenPlanHub to compare all providers side by side.

Common Questions About Tokens

How many tokens does a typical conversation use? A short chat message uses about 50-100 tokens. A code review might use 2,000-5,000 tokens. Processing a 100-page document uses roughly 30,000-50,000 tokens.

How can I reduce token usage? Keep prompts concise, use shorter context windows when possible, and batch similar requests together. Some providers offer caching for repeated prompts.

Does a larger context window always cost more? Not necessarily. Some providers include context window size in their subscription tiers. Others charge the same per-token rate regardless of context length. Compare plans on TokenPlanHub to find the best value.