Skip to content

Architecture Trade-off Analysis: Aegis Packet Engine

Low-latency systems engineering requires deliberate trade-offs between CPU utilization, latency determinism, code complexity, and memory usage. This document details the key trade-offs in the Aegis architecture.

1. Synchronization: Lock-Free SPSC Queues vs. Mutex-Guarded Queues

Metric / Feature Lock-Free SPSC Queue (Aegis) Mutex-Guarded Queue (Standard)
Typical Latency 10 - 50 ns 500 - 2,000 ns (under contention)
Tail Latency (p99) Deterministic (nanoseconds) Highly variable (scheduler/lock sleep)
CPU Utilization High (spin-waiting if polling) Low (threads sleep while blocked)
Code Complexity High (needs atomic memory ordering) Low (standard library wrapper)

Analysis:

  • The Cost of Locks: A mutex operation requires a system call (futex on Linux) if contested, forcing a context switch. A context switch takes 1.5 to 5 microseconds and invalidates L1/L2 caches.
  • Aegis Selection: We use lock-free SPSC (Single Producer Single Consumer) ring buffers. By keeping read and write heads in separate cache lines (padded with alignas(64)), we avoid cache bouncing and ensure that operations run in tens of CPU cycles.

2. Memory: Pre-allocated Buffer Pool vs. Runtime Heap Allocation (malloc)

Metric / Feature Contiguous Buffer Pool (Aegis) Runtime Allocation (malloc/new)
Allocation Latency O(1) Constant (~10-20 ns) O(N) Variable (microseconds to ms)
Memory Fragmentation Zero High (due to variable packet sizes)
Page Faults Zero (pages are touched/mapped at boot) Possible (on new block allocations)
Locality Excellent (contiguous memory blocks) Poor (pointer-scattered heap blocks)

Analysis:

  • Memory Fragmentation: Constant allocations of network frames of varying sizes (64B to 1500B) lead to heap fragmentation. The glibc allocator takes internal arena locks, which stalls packet processing.
  • Aegis Selection: We use a pre-allocated pool of fixed-size frames (2048 bytes, aligned to cache boundaries). Blocks are managed via atomic reference counters, allowing zero-copy sharing between workers and the output writer.

3. Storage: Flat Open-Addressed Hash Map vs. std::unordered_map

Metric / Feature Flat Bucket Array (Aegis) std::unordered_map (Standard)
Cache Locality Excellent (dense contiguous array) Poor (pointer-chasing node nodes)
Memory Overhead Fixed pre-allocated block High (overhead per node allocation)
Allocation on Insert Zero (uses pre-allocated nodes) Requires new heap node
Lookup Time O(1) with low cache-miss penalty O(1) with high pointer-chasing latency

Analysis:

  • Pointer Chasing: std::unordered_map uses bucket arrays pointing to linked lists of heap nodes. Accessing these nodes causes CPU cache misses as the memory addresses are scattered across the heap.
  • Aegis Selection: We use a flat bucket array with open addressing (linear probing or chained indices inside a contiguous block). This keeps collision chains close in memory, maximizing L1/L2/L3 cache hit rates.

4. Writer Output Architecture: Polling Array of SPSC Queues vs. Shared MPSC Queue

Metric / Feature Array of SPSC Queues (Aegis) Shared MPSC Queue
Producer Contention Zero (each worker has a private queue) High (workers contend on write pointer)
CAS Loops None (no atomic Compare-And-Swap) Required (for incrementing write index)
Writer Polling Round-robin polling of N queues Single read pointer polling
Instruction Cache Simple instructions Complex atomic loop

Analysis:

  • Multi-Producer Contention: In an MPSC (Multi-Producer Single Consumer) queue, multiple worker threads attempt to write to the queue simultaneously. This requires atomic Compare-And-Swap (CAS) instructions. Under load, CAS loops fail repeatedly (due to cache line invalidation between cores), causing latency spikes.
  • Aegis Selection: We use dedicated worker-to-writer SPSC queues. Each worker writes to its own queue, and the single writer thread polls them in a round-robin loop. This guarantees lock-freedom and zero contention on the write path.

5. Wait Strategy: Spin-Polling vs. Condition Variables

Metric / Feature Spin-Polling (Aegis) Condition Variable Wait
Response Latency Immediate (0-50 ns) 1.5 to 5.0 microseconds (kernel wakeup)
CPU Load 100% of pinned core Near 0% while idle
Throughput Jitter Zero High (unpredictable kernel scheduler delay)

Analysis:

  • Wakeup Latency: Condition variables put threads to sleep. When a packet arrives, the kernel must schedule and wake up the consumer thread. This latency is unacceptable in high-performance pipelines.
  • Aegis Selection: We use busy-wait spin loops on the fast path threads. For CPU efficiency when no packets are present, we execute the PAUSE instruction (on x86) to hint to the processor to save power and prevent pipeline flushes.