Context Window

The context window is the span of tokens a model can attend to in one call, covering the system prompt, the conversation, any retrieved documents, and the response being generated. Once input exceeds the window, earlier content has to be dropped or summarized, so the window size sets a hard limit on how much a model can hold in view at a time.

Context has a direct cost: every token in the window is billed and adds latency, so oversized prompts and stuffed context inflate the price of every request. Managing what goes into the window - trimming system prompts, retrieving only the relevant passages - is a central part of token optimization and a common place where spend quietly leaks.