Context & Memory

How agents remember, learn, and maintain awareness across sessions. 6-layer context assembly. Persistent memory extraction. Role-scoped isolation.

The Problem

AI agents are stateless. Every Claude session starts blank. But a real employee does not forget yesterday's work. For QUESTPIE Autopilot to work, agents need persistent memory, company awareness, role-scoped context, and session continuity -- all without reading each other's private data.

The filesystem already stores everything. The challenge is what to load into each agent's context window at session start, and what to persist when the session ends.

6-Layer Context System

Before every agent session, the Context Assembler builds a system prompt from six sources.

Layer 1: Identity (~2K tokens)

The agent's role definition. Loaded from /company/team/agents.yaml. This is the system prompt base and never changes within a session. It includes the role description, rules, team context, available tools, and filesystem scope.

Layer 2: Company State (~5K tokens)

A real-time snapshot of the company, scoped to the agent's role. Generated fresh before each session. Each agent role sees a different view.

Layer 3: Memory Store (~20K tokens)

The core innovation. Each agent has a personal memory file that persists across sessions. Memories are structured, searchable, and role-scoped. The Assembler ranks memories by relevance to the current task.

Memory content is ranked by relevance:

• High priority -- facts about the codebase, known mistakes to avoid, decisions related to the current task
• Medium priority -- recent sessions on other tasks, learnings file
• Low priority -- relationship notes for agents mentioned in the current task

Layer 4: Task Context (~15K tokens)

Everything about the current task: the task YAML, spec and plan documents, task history, dependencies, related messages, code context.

Token Budget Management

Each agent session has a total context budget of ~100K tokens (half of Claude's 200K context window). The remaining 100K is reserved for working memory during the session.

interface TokenBudget {
  total: number          // 100,000
  identity: number       // 2,000
  taskContext: number    // 15,000
  memories: number       // 20,000
  snapshot: number       // 5,000
  working: number        // 58,000
}

Memory File Structure

Each agent's memory lives in /company/context/memory/{agent-id}/:

/company/context/memory/
├── max/
│   ├── memory.yaml              # Structured memories
│   ├── learnings.md             # What I've learned
│   ├── relationships.md         # Working notes about teammates
│   └── sessions/
│       ├── 2026-03-22T10-00.md  # Session transcript summary
│       └── ...
├── sam/
│   └── ...

Memory Extraction

The Memory Extractor runs after every agent session. It uses Claude Haiku to analyze the session transcript and extract structured updates: new facts, decisions made, learnings, mistakes, and patterns.

async function extractAndPersistMemory(
  agentId: string,
  session: CompletedSession,
): Promise<void> {
  const memoryDir = `/company/context/memory/${agentId}`

  // 1. Generate session summary (~$0.001)
  const summary = await generateSessionSummary(session)
  await writeFile(`${memoryDir}/sessions/${session.startedAt}.md`, summary)

  // 2. Extract new facts, decisions, learnings
  const extractions = await extractMemoryUpdates(session)

  // 3. Merge into existing memory file
  const existingMemory = await parseYaml(`${memoryDir}/memory.yaml`)
  const updatedMemory = mergeMemory(existingMemory, extractions)
  await writeYaml(`${memoryDir}/memory.yaml`, updatedMemory)
}

// Total cost per session: ~$0.004
// Memory system costs: <$2/month

Memory Isolation Rules

Memory is private. No agent reads another agent's memory. Cross-agent information sharing happens only through task history and communication channels.

• No agent reads another agent's memory -- ever
• No agent reads session transcripts -- only summaries via task history
• CEO sees the most -- but never individual agent memories
• Cross-agent communication -- happens through channels and task history only
• Shared knowledge is readable by all -- within filesystem scope permissions

Summary