EN ▾

Uncensored Model Hubmodel_context_window_exceeded: Causes and Fixes

model_context_window_exceeded: Causes and Fixes

A context window error means your prompt plus the reply you asked for no longer fits in the model's memory. This article explains the limit on this API (100,000 tokens), how to measure your prompts, and code that trims history before it fails.

Updated

Glass container filled with glowing blocks, the overflow trimmed, representing a full context window

Key points

  1. The window is 100,000 tokens and counts the prompt and the completion together.
  2. Chat apps hit it because they resend the full history on every turn.
  3. Count or estimate tokens before sending, and keep a reserve for the reply.
  4. Trimming old turns also lowers cost, since input is billed at $0.15 per 1M tokens.

What model_context_window_exceeded means

The string model_context_window_exceeded is how some clients, SDKs and APIs label one specific failure: the text you sent, plus the room needed for an answer, is larger than the model can hold in one request. The wording of the message differs between tools, so read the full error body your client shows instead of relying on a name. The cause is always the same.

On this API the context window is 100,000 tokens, and that number covers the prompt and the completion together. If your prompt uses 95,000 tokens, at most about 5,000 are left for the reply. If you also set max_tokens to 8,000, the request asks for more than the window allows. The max_tokens field is optional, and when you leave it out, the reply simply uses whatever room is left.

A request that fails this way returns an error instead of a completion. Failed requests are not charged, so hitting the limit costs you nothing except the round trip. The fix is on your side: send less, or ask for less.

How big is 100,000 tokens?

A token is a chunk of text, often a short word or part of a longer one. For ordinary English prose a rough rule is one token per four characters, or about three quarters of a word. By that rule 100,000 tokens is around 75,000 words, which is a thick novel. Code, JSON, tables and non-English text split into more tokens per character, so the same character count can be much larger. Treat any rule of thumb as a planning number, not a guarantee.

The exact count comes from the model's own tokenizer. The API reports real usage in the response: the usage object lists prompt and completion tokens, and a streamed response sends usage at the end. That is the most reliable data you have, so log it.

ContentRough size
A chat message of two sentences40 to 60 tokens
One page of English text, 500 wordsabout 670 tokens
A 10,000-word documentabout 13,000 tokens
A 3 MB JSON dumpoften over 100,000 tokens, too big to send raw

The request body also has a size cap of 8 MB, which is a separate limit from tokens. Large inputs can fail on either one, and the token limit is almost always the first you reach.

Why chat apps run into the limit

The API holds no memory between calls. To continue a conversation, your code sends the whole message list every time. Each turn adds the user message and the reply, so the prompt grows with every exchange. A bot that works fine for an hour then starts to fail is the classic sign.

The same thing happens in agents. Every tool result is appended to the history, and one function that returns a big record can add thousands of tokens in a single step. Retrieval setups add a third source: if you paste twenty document chunks into each prompt, the base cost per request is already large before any history exists.

Before you fix anything, find out which of the three is growing. Print the prompt token count from usage after each call. If it climbs by roughly the same amount each turn, it is history. If it jumps after specific steps, it is tool output or retrieved text.

Trim history before you send it

The simplest reliable fix is a function that drops the oldest turns until the estimate fits a budget. Keep the system message always, and keep the latest user message always. The budget should leave room for the reply, so use something like 80,000 tokens for the prompt and let the rest be completion space.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.uncensoredmodelhub.com/v1",
    api_key="YOUR_API_KEY",
)

def estimate_tokens(text: str) -> int:
    return len(text) // 4 + 4  # rough: 4 characters per token plus overhead

def trim(messages, budget=80_000):
    system = [m for m in messages if m["role"] == "system"]
    rest = [m for m in messages if m["role"] not in ("system",)]
    total = sum(estimate_tokens(m["content"]) for m in system + rest)
    while total > budget and len(rest) > 1:
        dropped = rest.pop(0)  # oldest turn goes first
        total -= estimate_tokens(dropped["content"])
    return system + rest

history = [{"role": "system", "content": "You are a helpful assistant."}]

def ask(user_text):
    history.append({"role": "user", "content": user_text})
    resp = client.chat.completions.create(
        model="uncensored",
        messages=trim(history),
        max_tokens=2000,
    )
    reply = resp.choices[0].message.content
    history.append({"role": "assistant", "content": reply})
    print(resp.usage.prompt_tokens, resp.usage.completion_tokens)
    return reply

Keep user and assistant turns in pairs when you can; dropping one half of a pair can leave the model confused. Tool messages are similar: a role: "tool" message must stay with the assistant turn that requested it, so drop them together. Because the estimate is rough, a 20 percent gap between your budget and the real window is a comfortable safety margin.

Better options than dropping old turns

Summarize, then continue

When the history passes a threshold, send the oldest half to the model with an instruction to summarize it in a few hundred words, then replace those turns with the summary as a system or user message. You lose detail but keep the thread. Run this step outside the live reply path so the user is not waiting for it.

Pin the facts that must survive

Keep a small block of durable notes, such as names, decisions and settings, and re-insert it at the top of every request. This works better than hoping an old turn is still in view.

Shrink what you feed in

Truncate tool results to the fields that matter. Retrieve fewer chunks, and cut each one to the relevant paragraphs. For a very long document, split it into parts, process each part, and combine the results in a final short call.

Set max_tokens deliberately

If answers are normally short, cap max_tokens at a value that fits your product. A cap protects the budget, but a cap that is too low cuts replies mid-sentence, so look at your real answer lengths in the logs first.

Trimming saves money as well

Input tokens cost $0.15 per 1M, output tokens $0.50 per 1M, billed from prepaid credit. Because chat resends the history, cost per conversation grows faster than its length. Take a chat where each exchange adds 500 tokens to the history and the conversation runs 100 turns.

  • Without trimming, turn n sends 500 x n tokens. The sum over 100 turns is 500 x 5,050 = 2,525,000 input tokens, which costs 2.525 x $0.15 = $0.379.
  • With a 20,000-token cap, the first 40 turns are the same (500 x 820 = 410,000 tokens). The remaining 60 turns send 20,000 each, which is 1,200,000. The total is 1,610,000 tokens, or 1.61 x $0.15 = $0.242.

The cap saves about a third here, and the gap widens as chats get longer. At the extreme, a full 100,000-token prompt costs 0.1 x $0.15 = $0.015 per request. That is cheap on its own, but it adds up in a bot that serves many users. Pricing details and top-up bonuses are on the pricing page, and credit never expires.

Handling the error in code

Even with trimming, plan for a failure. Catch the API error, check whether the message points at the context length, shrink the request, and retry once. Do not retry the identical request, since it will fail the same way.

from openai import BadRequestError

def safe_ask(messages, budget=80_000):
    try:
        return client.chat.completions.create(
            model="uncensored", messages=trim(messages, budget)
        )
    except BadRequestError as err:
        if "context" in str(err).lower() and budget > 10_000:
            return safe_ask(messages, budget // 2)
        raise

Check the actual status code and message your client receives for an oversized request and adjust the test; the string match above is a starting point. Rate limits are separate: HTTP 429 with rate_limited or concurrency means too many requests, not too many tokens, and 429 with Retry-After: 5 means a temporary overload. The docs list the request fields, the explainer on uncensored models covers what the model is, and you can get a key to test any of this with real usage numbers.

Questions and answers

How many tokens can I send in one request?

The window is 100,000 tokens, shared between your prompt and the completion. Whatever the prompt uses is no longer available for the reply.

Am I charged when a request exceeds the window?

No. Failed requests are not charged. You pay per token only for requests that complete.

Do I have to set max_tokens?

No, it is optional. Without it, the completion can use the remaining room in the window. Set it if you want a hard cap on reply length and cost.

How do I count tokens exactly?

Read the usage object in each response, which reports real prompt and completion tokens. For estimates before sending, a four characters per token rule works for English.

Your key is one form away

Create an account, copy the key, change the base URL. That is the whole setup.

Get API key