Building Valo: ContextVar Logging — Thread-Safe Context Without the Chaos

You can't debug what you can't trace.

Today we implemented Valo's structured logger with ContextVar-based context propagation — the exact pattern Anthropic's Sentinel uses to keep production logs traceable across thousands of concurrent AI workflows.

The Problem: Lost Context in Concurrent Operations

Standard Python logging works great for single-threaded apps, but add concurrency and you lose the thread (literally). Good luck debugging that in production.

Traditional Solutions (And Why They Fail)

Approach 1: Pass request_id everywhere

Every function signature explodes. Functions that don't care about logging now need request_id. Refactoring nightmare.

Approach 2: threading.local()

Breaks with async/await. Each coroutine shares the thread, so contexts collide.

Approach 3: Global variables

Don't. Just don't.

The Solution: ContextVar

Python 3.7 introduced contextvars — thread-safe and async-safe context storage.

Key properties:

• Thread-safe: Each thread has isolated context
• Async-safe: Each coroutine has isolated context
• Copy-on-write: Child contexts inherit parent values
• Automatic cleanup: Contexts disappear when execution ends

Implementation: Three Core Components

1. ContextVar Storage

from contextvars import ContextVar
from typing import Any

_log_context: ContextVar[dict[str, Any]] = ContextVar("log_context", default={})

This is the magic. Each execution context gets its own isolated dictionary.

2. JSON Formatter

Injects context from ContextVar automatically, along with extra fields from log calls.

3. Context Management Functions

Simple API: set_log_context() and clear_log_context()

Why JSON? CloudWatch and Beyond

Plain text logs are human-readable but machine-unparseable. JSON logs are both.

{
  "timestamp": "2026-06-08T12:01:09.776Z",
  "level": "INFO",
  "request_id": "req-123",
  "component": "orchestrator",
  "match_id": "12345",
  "message": "Processing match"
}

CloudWatch Insights query: Instant request timeline. Try doing that with plain text logs.

Usage in Practice

Set context once at request entry point. All subsequent logs include context automatically. No parameter passing. No global variables. Just works.

Quality Metrics

• 67 tests passing (15 new logger tests + 52 previous)
• 96% overall coverage (structured logger: 100%)
• Zero mypy errors (strict mode)
• Zero ruff errors

Traceable. Parseable. Production-ready.