Context & Memory

Models don't "remember" conversations. Every time you message them, they re-read EVERYTHING that was said — up to the window limit. Anything beyond the window disappears.

How a model "sees" a conversation

Imagine the context window as a fixed-length scroll. Each new message is written at the tail. When the scroll fills up, the head gets cut — the first messages fall off and are forgotten permanently.

   [ forgotten ]  ←── outside the window
  ┌───────────────────────────────────────────┐
  │  Message 1: project setup        ── drops │
  │  Message 2: requirements         ── drops │
  ├───────────────────────────────────────────┤
  │  Message 23: login clarification          │
  │  Message 24: validation discussion        │  ← context window
  │  Message 25: last message (now)           │     (what the model SEES)
  └───────────────────────────────────────────┘

Typical context window sizes

• GPT-4 standard: ~128K tokens (~96,000 EN words)
• Claude Sonnet: 200K tokens, optionally 1M
• Gemini 1.5: up to 1M tokens
• For perspective: 200K tokens ≈ a 500-page book.

Persistent memory (outside conversation)

Some AI tools (Claude Code, ChatGPT with memory, Cursor) have "external memory" — files where they write facts about you or your project, so they can re-read them in future conversations.

• Short-term memory = current context window. Erased when you close the conversation.
• Long-term memory = persistent files (e.g. MEMORY.md, CLAUDE.md). Loaded at every new conversation.

RAG — when context doesn't fit

RAG = Retrieval-Augmented Generation. Plainly: instead of sending all your docs in the prompt (won't fit), you store them in a special database and — for each question — extract ONLY the relevant fragments and add them to context.