Every chat feels like a conversation. It isn’t. Your AI assistant is a goldfish holding a photograph of you. It forgets you the instant you hit Enter. Then a script slides the photograph, the entire history, and a small mountain of standing instructions back in front of it, and it reads the whole pile again before answering your next question.

That is not a bug. That is the architecture. Nobody put it in the marketing.

The stack that gets re-sent every turn

When you type a question into any modern chat assistant (Copilot, Claude, ChatGPT, Gemini, take your pick), here is what actually gets shipped to the model on your behalf:

1. The system prompt. The vendor’s operating instructions for the assistant. Usually several thousand tokens. You are not shown it, are not asked about it, and cannot turn it off. It rides along regardless.
2. Your persistent memory. Notes the assistant was told to remember about you: name, role, preferences, that offhand comment you made in March. You did not curate this list. It curated you.
3. Tool and skill descriptions. Every action the assistant might take (search the web, run code, open a file, call an API) shows up as a schema block. One tool is roughly 100 to 300 tokens. Many assistants ship with dozens, on the theory that you might, someday, use one of them.
4. Custom instructions. Personal or repository-level rules like “always use TypeScript.” Set once, forgotten forever, still shipping.
5. The chat history. Every previous message in this thread, from your first “hey” to the assistant’s last reply.
6. Your new message. The question you actually asked.

All of that goes together as one payload. The model reads it start to finish, generates a reply, and forgets you again. You type your next question. Everything above happens again. From scratch.

Why “from scratch” is not a metaphor

Language models are stateless. There is no ongoing session in the model’s head the way there is in yours during a conversation. Each turn is a cold read by a stranger who happens to have your entire file open.

The chat feeling of continuity is a stage trick. The wrapper around the model, the app you are actually using, does all the bookkeeping. It appends your turn to the history, re-ships the whole pile, catches the reply, pastes the reply back, waits for your next input. The model itself remembers nothing between turns. It never has.

Close the tab and reopen it. The wrapper reconstructs your session and re-ships it. The model does not go “oh, welcome back.” The model has never met you. It will never meet you. You are pen pals with someone who burns each letter after reading it.

Three costs you pay for the re-read

Money. Providers bill per token processed. Every turn processes the entire pile again, so per-turn cost climbs as the conversation grows. Prompt caching softens this. Vendors cache stable prefixes and re-bill them at a fraction of full price. Caching is not free, does not cover everything, and expires without warning. But it exists, and they will remind you it exists when you complain about the bill.

Latency. More tokens to read means more time before the reply starts arriving. A 30,000-token context takes longer to first-token than a 3,000-token context. This is why the assistant that was crisp on Monday is thoughtful on Friday.

Quality. The one nobody talks about. Every model has a context window, a hard cap on how many tokens it can hold at once. As the pile grows, three things happen, none of them in your favor:

• Older messages get pushed further from the current question. The model pays less attention to them. “Attention” here is not a metaphor. It is a specific mathematical operation, and older tokens get less of it.
• Once the window fills, the wrapper starts silently dropping the oldest messages to make room. “Silently” is doing a lot of work in that sentence. Your assistant now literally cannot see what you said 40 turns ago, and it will not tell you it cannot see it.
• Even before you hit that ceiling, model performance degrades on long inputs. Attention thins out in the middle of very long contexts. Researchers call this “lost in the middle.” The model calls it nothing, because the model cannot tell.

Long conversations get dumber. That is not perception. That is mechanics. Your assistant is not tired. Your assistant is buried.

What actually eats the window

Most users assume their chat history is the biggest thing in the payload. Most users are wrong. On a real audit of a working developer machine, the ranking was:

1. System prompt and vendor overhead. Several thousand tokens. Fixed. Non-negotiable.
2. Tool and MCP schemas. Often the single largest variable block, especially with multiple servers connected. 5,000 to 15,000 tokens is common. This is what you paid for when you clicked “add MCP server” on that GitHub link six months ago and forgot about it.
3. Custom instructions and memory. 1,000 to 5,000 tokens, growing quietly. Nothing dramatic, just steady sediment.
4. Chat history. Variable, but on short sessions often smaller than any of the above.

Your question, the thing you actually care about, is usually the smallest item in the payload. You are footnotes in your own conversation.

What to do about it

Two things, in order.

Stop treating conversations as free. They are not. If a session is running long and getting weird, start a new one. You are not offending the assistant. You are giving it a smaller pile to read. It will thank you by being smarter and faster, which is what it should have been in the first place.

Audit your standing overhead. Everything in items 1 through 4 above rides along on every turn whether you use it or not. Old memory notes you forgot about. MCP servers you connected six months ago and never touched. Custom instructions listing frameworks you no longer use. All of it is rent you pay per question. Cutting it saves money on cached surfaces and reclaims context window on every surface. The window is the thing that actually matters. Nobody sells you a bigger one because you asked nicely.

The takeaway

Your AI assistant is not having a conversation with you. It is being handed a growing stack of paper before every question and asked to answer as if it just walked in. The stack is what costs you money, slows you down, and eventually crowds out the room for a good answer.

Understand the stack. Trim it. Get better answers.

The alternative is to keep paying premium prices for premium confusion.