AI Network & Security Edge

The inspection pipeline every request crosses before it ever reaches VRAM — WAF, AI guardrails, and hardware-accelerated offload.

Article 7 starts the request path at the edge / POP: ingress, TLS, then into the cluster. This deep-dive stays one hop earlier and one layer deeper — what inspects and admits that request before an inference engine (or the KV-aware balancer in Article 7 §4) ever sees it. If you skipped straight here from the map, treat Article 7 as the overview and this page as the perimeter detail.

1. Three layers on one perimeter

Think of admission as a stack, not a single box:

• Front-end network — the familiar perimeter: WAF signatures, L4/L7 firewall rules, TLS termination, and a load balancer that may still be geography- or health-based at this hop. This is the same job you already own for any production API.
• AI guardrails — prompt and response filters that are content-aware: jailbreak patterns, PII, policy-blocked topics, toxic output. They are detection systems with a real false-positive rate — tune them like IDS rules, not like a binary allowlist you set once.
• DPU / SmartNIC offload — move crypto, some inline inspection, or telemetry parsing onto programmable NIC hardware so host CPU (and GPU) cycles stay on inference. Same tradeoffs as any inline appliance: throughput, added latency, and failure mode if the offload path dies.

2. Where this sits relative to Article 7's balancers

Article 7 §4 really has two L7 hops — a global balancer (region pick) and a cluster balancer (node pick on cache + queue). The perimeter in this article sits in front of both: terminate TLS and run coarse WAF at the edge → optional guardrail service (inline or sidecar) → then those KV-aware L7 decisions → GPU node. Guardrails that run after the smart balancer waste a cache-aware placement on a request you are about to reject. Guardrails that run too early, before you know model/tenant routing, can force a second inspect hop. Pick one choke point and measure it.

Output filtering is the awkward sibling: the response streams back through the same perimeter. Streaming guardrails must decide whether to buffer (hurts TTFT) or inspect token windows as they pass (risks leaking a few bad tokens before a block). That is a product/security tradeoff with a direct latency cost.

3. The latency tax against TTFT

Article 5 defined TTFT as the wait until the first token. Everything on this perimeter is before prefill starts — so WAF + guardrail + offload latency lands straight in the user's "is this thing awake?" number. Budget it the way you budget firewall inline latency on a low-RTT path: know the p50/p99 of the inspect hop, and decide what fraction of your TTFT SLO you are willing to spend before a GPU ever touches the prompt.