Skip to content

Product Requirements Document (PRD): Aegis Packet Engine

1. Executive Summary

Aegis is a high-performance, low-latency, stateful network packet processing and Deep Packet Inspection (DPI) engine. It is designed to run on multi-core systems, ingest packets at multi-gigabit rates, track flows, reassemble TCP streams, classify application-level protocols, evaluate blocking rules, and output filtered traffic with minimal overhead.

This document establishes the product vision, scope, functional requirements, and performance targets required to elevate the baseline project to a production-grade systems engineering artifact.

2. Problem Statement & Motivation

Network infrastructure components (such as firewalls, intrusion detection systems, and trading feed handlers) require deterministic performance. The baseline Packet_analyzer is a diagnostic tool using lock-based synchronization, unbounded dynamic allocations (e.g., copying raw data into std::vector, generating std::string representations of IPs/MACs on every packet), and simple substring searches. Under load, these design choices cause: 1. Thread Contention: Multiple threads blocking on standard mutex-guarded queues. 2. Memory Fragmentation: Constant heap allocation/deallocation per packet. 3. Cache Inefficiency: Non-contiguous packet buffers and pointer-heavy structures. 4. Functional Gaps: Inability to handle fragmented TCP handshakes, out-of-order packets, or advanced IP/CIDR rule matching.

Aegis solves these issues by shifting to a zero-copy, lock-free, cache-aware architecture.

3. Scope & Non-Goals

In-Scope

  • Zero-Copy Ingestion & Parsing: Extracting protocol layers without memory copies of packet payloads.
  • Lock-Free Concurrency: Single-producer single-consumer (SPSC) lock-free ring buffer arrays.
  • TCP Stream Reassembly: Basic reassembly of TCP payloads for handshake and SNI extraction.
  • Advanced Rule Evaluation: CIDR range matching, exact IP match, and domain/SNI matching.
  • High-Resolution Benchmarking: Packet processing latency tracking (p50, p95, p99) and throughput metrics.
  • Platform Portability: Optimized for Linux systems programming (thread affinity, memory alignment) with a functional Windows fallback.

Out-of-Scope (Non-Goals)

  • Full Layer 7 Gateway: Aegis is an inspection and filtering engine, not a proxy or proxy-server (like Envoy).
  • Hardware Acceleration (DPDK/PF_RING): While DPDK would be used in a real trading environment, we will simulate DPDK-like behavior in user space using high-speed files and lock-free queues, to maintain cross-platform compile capabilities without proprietary drivers.
  • GUI Dashboard: The UI will be a CLI and terminal-based real-time stats output.

4. Requirements

4.1 Functional Requirements

  1. Ingestion Modes:
  2. Offline Capture: Read from standard PCAP files.
  3. Live Capture: Capture live packets from a network interface (using Libpcap wrapper).
  4. Synthetic Generator: Command-line suite to create heavy load profiles.
  5. Stateful Flow Tracking:
  6. Track flows using a 5-tuple key: (SrcIP, DstIP, SrcPort, DstPort, Protocol).
  7. Support bi-directional flow associations.
  8. Expire stale flows automatically using LRU or time-based expiration to prevent memory exhaustion.
  9. Deep Packet Inspection:
  10. Extract Server Name Indication (SNI) from TLS Client Hello.
  11. Extract Host field from HTTP GET/POST headers.
  12. Parse DNS queries.
  13. Rule Matching:
  14. Block by source/destination IP, CIDR subnet (e.g. 192.168.1.0/24), protocol, port, or SNI domain.
  15. Support dynamic rule reloads without restarting the engine.
  16. Observability:
  17. Output per-thread metrics (packets processed, queue drops, average latency).
  18. Output flow stats, application breakdowns, and dropped packet logs.

4.2 Non-Functional Requirements (NFRs)

  1. Performance:
  2. Throughput: Target maximum throughput scaling linearly with the number of worker cores, exceeding the baseline multi-threaded engine performance (measured during benchmarking).
  3. Latency: Target sub-microsecond p50 processing latency per packet, and highly bounded tail latency (measured during benchmarking).
  4. Lock-Freedom: Lock-free queues on the critical data path.
  5. Memory Efficiency:
  6. Zero Heap Allocations on Data Path: Use pre-allocated packet and flow pools.
  7. Cache Alignment: Align packet buffers and thread metadata to 64-byte cache lines to avoid false sharing.
  8. Portability & Reliability:
  9. Compile using C++17 with strict compiler flags (-Wall -Wextra -Werror -pedantic).
  10. Clean compilation under GCC, Clang, and MSVC.
  11. Zero memory leaks (verified via Valgrind / AddressSanitizer).

5. Success Metrics

  • Latency Reduction: > 10x reduction in p99 packet latency compared to the baseline.
  • Throughput Scaling: Throughput should scale linearly with worker threads up to the physical CPU core limit.
  • Zero Allocations: Memory profiling (e.g. using heaptrack or valgrind --tool=massif) must confirm zero heap allocations during the packet processing loop.