DDoS Defense

The Great Separation: Why Your Firewall is the Wrong Tool for Ingress

Hervé Hildenbrand

— 3 min read

A note before we begin: What follows is my perspective forged from decades of operational scars. It's not gospel it's one engineer's hard-won opinion. Take what's useful to you and feel free to challenge the rest.

I stood in the war room during our Q3 stress test. The goal: Survive a 10Gbps SYN flood. The result: The service died at 500Mbps.

It didn't even reach 1% of our target capacity.

The load balancers showed 0.2% CPU utilization. They were bored. The expensive "next-generation" firewall pair was screaming at 100% CPU dropping valid packets because its memory was full.

This experience taught me that the prevailing network security orthodoxy has transformed into a critical liability. We call this pattern "Firewall Everything."

It stems from a dangerous misconception: some engineers think traffic direction is symmetrical. They think the tool used to control traffic leaving the network should be the same tool used to control traffic entering it.

The Reality of State

I know what you're thinking: "The SYN flood was designed to kill state tables, of course it failed."

You are right. But in modern high-scale apps "normal" traffic often looks like an attack to a stateful device. Microservices, WebSockets and rapid-fire API calls create a level of chaos that stateful firewalls struggle to track even without a DDoS.

When you place a general-purpose stateful firewall in front of your Ingress path, you are not building "Defense in Depth." You are building a performance suicide pact. Here is why:

1. The "State Police" Problem

Imagine a stadium designed to hold 50,000 people. You have 50 high-speed turnstiles (Load Balancers) to process the crowd. But then, you place a single security guard (Firewall) on the sidewalk before the turnstiles.

This is the stateful trap. A firewall acts as State Police. It tracks every sequence number and flag in the kernel-level state table (conntrack). During a flash sale or an attack, this table becomes a single point of failure.

2. The Asymmetry Trap

Statefulness breaks the promise of dynamic routing. If a network link fails, BGP re-routes traffic in milliseconds. Routers are happy. Firewalls are not.

If traffic enters Firewall A but tries to leave through Firewall B due to a routing change, the connection dies. Firewall B sees a return packet without a handshake record and drops it.

3. The "Multiplexing" Shield

This is the hidden killer most architects miss.

  • The Firewall (1:1 Mapping): If 10,000 clients connect, the firewall passes 10,000 TCP connections to your backend. Your servers waste CPU handling 10,000 TLS handshakes.
  • The Load Balancer (Multiplexing): A load balancer accepts 10,000 connections at the edge. It routes those requests over 50 "warm" persistent connections to the backend.

3 Signs You Are Suffering from the "Ingress Paradox"

If you aren't sure if your architecture is at risk, look for these three symptoms:

  1. The CPU Inverse: Your Firewall CPU spikes during traffic surges, while your Load Balancer CPU remains flat.
  2. The Fear of Asymmetry: Your network team is afraid to enable ECMP (Equal-Cost Multi-Path) routing because it might break the firewall clusters.
  3. The "Phantom" Outage: Users report timeouts but your backend server logs show nothing (because the packets never made it past the state table).

The New Standard: A Decision Matrix

"No Firewall" does not mean "No Inspection." It means choosing the right tool for the layer. Here is the framework:

  • Layer 3/4 (Volumetric): Use BGP Flowspec or Router ACLs.
  • Layer 7 (Application): Use WAF (on Load Balancer).
  • Egress (C2 Prevention): Use Stateful Firewalls.

When This Rule DOESN'T Apply

Senior engineering is always about context and not absolutes. There are times when "Firewall First" is still correct:

  1. Small Scale / SME: If your total bandwidth is under 1Gbps and you have 50 users a single Next-Gen Firewall is cost-effective and simpler to manage. Complexity is the enemy here. Of course most of the traffic here is outbound.
  2. Non-Web Protocols: If you are ingesting raw TCP streams (like proprietary IoT protocols) that a WAF cannot understand a stateful firewall is potentially your only line of defense.
  3. Compliance Mandates: Certain strict environments (legacy Gov/Banking) explicitly mandate a "Firewall" at the edge. In this case, you comply but you oversize that box significantly or place it in "Transparent Mode".

The Verdict

Stop treating the firewall as a magic security blanket. Ingress: Use Load Balancers to survive the flood. Use WAFs to inspect the payload. Egress: Use Firewalls to stop the exfiltration.

Read more