A million-token context window: when it actually helps
A larger window can bring related documents together. Testing the task shows whether the extra context improves the answer.
An investigation depends on a contract rule, an exception in a manual and a change recorded in a decision log. If the system sees only one document, it may answer correctly about that passage and incorrectly about the whole situation.
This is the kind of task where a larger window deserves testing. Gemini’s long-context documentation describes models with a million or more tokens and directs readers to each model’s specific limit.
Capacity does not guarantee correct use
The context window limits the material available in a run, subject to the model and API rules. It is not permanent memory of everything the company has submitted. Nor does it ensure every passage receives the right emphasis in the answer.
Before enlarging the input, specify the relationships the answer must establish. “Summarise these files” is broad. “Identify manual exceptions that contradict the contract rule and cite the passages” offers a verifiable outcome.
Compare three ways of supplying information
Selected passages: provide only relevant documents or excerpts. This may suffice for a localised question when selection is reliable.
The complete collection: supply related material for cross-document analysis. This may help when dependencies are hard to anticipate, provided the material fits the limits and operating requirements.
Progressive retrieval: let the system find more information as the investigation proceeds. It needs a record of what was consulted and what remains missing.
These approaches need not compete in every situation. A workflow can begin with search and expand reading when an important relationship emerges. Choose around the question rather than the desire to fill the window.
Test mistakes that would matter
Prepare questions with answers checked by someone familiar with the documents. Include details in different sections, contradictory versions and a question whose answer is absent. Check whether the system reports that absence instead of filling the gap with an assumption.
Request precise references and inspect them. A citation that exists but does not support the conclusion is still wrong. Compare correctness, time and task cost across the approaches using the same questions.
Organise the material before adding more
Identify document, date and version. Remove duplicates and irrelevant content while preserving the origin of selected passages. For confidential files, decide the processing route and access rules before loading them.
Use the large window to address an observed limitation. If a smaller input gives the same result with less operating effort, there is no benefit in sending the rest simply because it fits.
Continue with what operating an open-weight model involves.
About the author
Tiago F SantiagoComments
No comments yet
Share a question or an experience related to the article.
Related articles
