AI & Tech

The Number Missing From the Checkable Files

A compressed summary of a long AI conversation contained a number absent from the project's checkable files; that leads into six things Anthropic's official guide says a summary must not lose, and a separate six-part request written by the author that worked only halfway.

2026.09.03 · By dvdmaru · ~7 min read · 1,660 words

本文另有中文版:Claude 聊到一半忘了你講過的話:官方文件列了六件摘要不能丟的事

A compressed summary of a long AI conversation contained the number 95,000. The client project’s files that could actually be checked did not contain that number. The figure that did check out, 77,052, was sitting in those same files the entire time.

How does a summary come to hold a number absent from the checkable files? This piece is about compaction. A long conversation gets folded into a summary when it has to be, and after that, Claude Code uses the summary to carry the work forward. The documentation says only that early details may be lost, not that they will be.

Anthropic’s official prompting guide for Claude Fable 5.1 gives one whole section to this problem and lists six things a summary must not lose.

As a Conversation Nears Its Limit, Claude Code Clears Old Output Before It Summarizes

According to the Claude Code documentation, when a conversation approaches the limit of its context window, Claude Code first clears out older tool outputs. If needed, it then summarizes the conversation. Requests and key code snippets are preserved, the documentation says, but “detailed instructions from early in the conversation may be lost” (Claude Code documentation, read September 2026). The same documentation recommends putting rules meant to last the whole project into a settings file, rather than counting on the conversation history to carry them.

The subject of that sentence is the summary, not the model. This piece only has access to two documents — Anthropic’s prompting guide for Claude Fable 5.1 and the Claude Code documentation — and neither says whether other AI products handle long conversations the same way.

The Official Guide Lists Six Things a Summary Must Not Lose

Anthropic’s prompting guide for Claude Fable 5.1 (read September 2026) carries no date of its own. One section addresses this directly, under a heading about telling the model what a compaction summary must preserve.

Its opening line says Claude Fable 5.1 “responds well to being told explicitly what its summary must retain”. The next line adds: “Server-side compaction already does this.” The instruction that follows is meant for people compacting a conversation on the client side themselves, not a rule the guide says every user needs to add.

The six things are:

#What the guide says to keepIn an ordinary chat, that means
1Difficulties or problems that came up, and how they were handled or resolvedThe snags you worked through together
2Options that were raised, tried, or set aside, and whyIdeas you considered and didn’t take
3Anything asked for, decided, agreed, ruled out, or set as a preference, constraint, or boundary — “stated exactly”The rules you and it settled on
4Exactly where things stand nowHow far along the work is
5Anything still open, unresolved, promised, or expected to happen nextWhat’s still owed
6Details that would be hard to reconstruct — names, numbers, dates, exact wording, links, or references — “kept exactly”The names, numbers, dates, and links you gave it

The guide adds one line about the whole list: “Be complete on these even at the cost of length; keep everything else concise.”

It also asks for the two voices in a conversation to be weighted differently. What a user said, asked for, or established should stay close to their own words. The model’s own explanations can be compressed much further, so long as none of the six items above are dropped. A separate line elsewhere in the guide names the cost of skipping this: “Client-side compaction summaries drop constraints, decisions, or exact details”.

A Custom Six-Part Request Got Six Parts Back, Done Only Halfway

On the evening of September 2, 2026, the author added a request to a personal settings file: that any compaction summary come with six sections attached. This was the author’s own version, not a copy of the guide’s six items — it left out the guide’s first two items and added two of its own, one asking that any number be flagged with its source rather than trusted outright, and one asking that any decision made on the author’s behalf come with the cost of getting it wrong.

That same evening, an open conversation was manually compacted: the author sent the compaction command directly, rather than it happening automatically as the conversation filled up. The summary that came back did have all six sections at the end. Eight file paths were listed, three direct quotes were kept, and three decisions made on the author’s behalf came with a stated cost. The section on numbers listed a source for each figure and marked it as needing to be rechecked.

But that same section still carried the old values in parentheses next to the flag, so the instruction had been followed only halfway. The summary itself flagged this as a gap.

This is one observation, not a documented behavior. Whether a personal settings file like this one takes effect on every compaction is not something either document says, and this observation covers only a manual compaction, not an automatic one.

Compacting Early Is Not Necessarily the Cheaper Choice Anymore

Cache reads got cheaper with Fable 5.1 (OpenAI Designated Astra Critical; Anthropic Said Its Models Fall Short of Its Next Tier), and the guide gives that as its reason. It states that “compacting early to save cost may no longer be the right cost-intelligence tradeoff” and suggests testing later compaction points instead. It does not say how much later.

The Claude Code documentation names one more option: adding a “Compact Instructions” section. How to write one is outside the scope of this piece — the operational version, linked at the end, covers that. The guide’s six items and Claude Code’s own clear-then-summarize process are described in two separate documents; neither says whether Claude Code’s built-in summarizer runs on the guide’s six items.

Say What Can’t Be Forgotten Before You Start; Recheck the Numbers After

There are two things a reader can do here. The first is to say, before a long piece of work starts, what cannot be forgotten — the rules that were settled on, in plain words, along with names, numbers, dates, and anything still owed. The second is to treat a summary’s numbers as something to check, not something to trust: after a compaction, go back to the original file for anything that matters.

Among the six items, the two the author worries about most are exact wording and numbers. Once either gets rewritten or gotten wrong, it is usually hard to reconstruct from the summary alone. That is the author’s own ranking of the risk, not a statistic from Anthropic.

Back to where this started: the summary says 95,000, and the checkable source files say 77,052. That is one case, not an error rate. But if a conversation carries on from the summary, the wrong number goes along with it, which is why a number that matters is worth checking against the original once.

Further reading: the Claude Code operator’s version

Frequently Asked Questions

Q: Why does Claude seem to forget what I said earlier in a conversation? This piece is about an information gap that compaction can create, not a verdict on how well any model remembers. Claude Code’s own documentation describes a process: as a conversation nears the limit of its context window, Claude Code clears older tool outputs first, and summarizes the conversation only if it still needs to; after that, Claude Code uses the summary to carry the work forward. Requests and key code snippets are preserved, the documentation says, and early, detailed instructions can end up lost. The subject there is the summary, not the model. This piece only has access to what Anthropic’s guide for Claude Fable 5.1 and the Claude Code documentation say; whether other AI products behave the same way was not checked here.

Q: What are the six things the official guide says a summary must keep? Anthropic’s official prompting guide for Claude Fable 5.1 lists six things a compaction summary must retain: difficulties that came up and how they were handled; options that were raised, tried, or set aside, and why; anything that was asked for, decided, agreed, ruled out, or set as a preference, constraint, or boundary, kept close to the original wording; exactly where things stand now; anything still open, unresolved, promised, or expected to happen next; and details that would be hard to reconstruct — names, numbers, dates, exact wording, links, or references — held onto precisely. The guide says to be complete on these six even at the cost of length, and to keep everything else concise.

Q: Is there a way to tell it what not to lose? The Claude Code documentation names one option: a section called Compact Instructions; how to write one is outside this piece. A more general habit works too — before starting a long task, say plainly what must not be forgotten: the rules you have agreed on, in your own words, along with names, numbers, dates, and anything still owed.

Q: Can the numbers in a summary be trusted? Not automatically. In one case, a compaction summary produced the number 95,000; a client project’s checkable files did not contain that number, and the figure that did check out, 77,052, was sitting in those files the entire time. After a conversation is compacted, the safer habit is to check any number that matters against the original files rather than trust the summary’s version of it.