Why Tokio, not OS processes

The most important architectural decision in Trubase DB is the choice to run every backend as a Tokio async task instead of an OS process. This single decision is what enables millions of backends per node, true scale-to-zero, and millisecond branching. Here's the reasoning.

Legacy PostgreSQL uses a 'postmaster' process that forks a new OS process for every incoming connection. Each process gets its own address space (~10–30MB of RAM), is scheduled by the OS kernel's CFS scheduler, and communicates with other processes through POSIX shared memory (shmget/shmat). This model made sense in 1996 when PostgreSQL was designed for a handful of concurrent users on a single machine.

But today? A single AI agent platform might need 100,000 concurrent backends. At 30MB per process, that's 3TB of RAM just for connection overhead. The OS kernel's process scheduler was designed for hundreds of processes, not millions. Context switching between millions of OS processes would saturate the CPU with kernel overhead.

Tokio is a multi-threaded async runtime for Rust that uses a work-stealing scheduler. Instead of OS processes, every backend is a tokio::spawn async task. These tasks run on a shared pool of OS threads (typically one per CPU core). When a task is waiting on storage or network I/O, it yields the CPU — zero waste. When one core finishes its work, it steals tasks from another core's queue — perfect load distribution.

Each task weighs a few KB of state machine memory because Rust async functions compile to state machines, not stack frames. The compiler transforms the entire Postgres query loop — read packet → parse SQL → plan → execute → write response — into a Rust Future, a zero-allocation state machine. No garbage collector. No runtime overhead.

The math speaks for itself: on the same hardware where legacy Postgres handles ~1,000 connections (each a 10-30MB OS process), Trubase DB multiplexes millions of backend tasks. That's not an incremental improvement — it's a categorically different architecture. And it's what enables backend per user, per agent, per workflow.

This architectural choice is also what enables true scale-to-zero. An OS process requires teardown — closing file descriptors, unmapping memory, signaling the postmaster. A Tokio task just completes. Its memory is instantly available for the next task. No container shutdown. No VM drain. The task ends and the memory is reclaimed in nanoseconds.