Reverse Proxies and Load Balancers: NGINX, HAProxy, and Envoy in Practice

What a reverse proxy actually does

A reverse proxy sits in front of your application servers and accepts client connections on their behalf. Unlike a forward proxy, which acts on behalf of the client, a reverse proxy acts on behalf of the server. It terminates the incoming connection, decides which backend should handle the request, forwards it, and returns the response.

This single hop gives you a place to handle TLS termination, request routing, rate limiting, buffering, and observability without changing application code.

L4 versus L7: know the layer you operate at

Layer 4 load balancers work on TCP and UDP. They look at source and destination IP and port, not the content of the request. HAProxy is strong here, though it also does L7.

Layer 7 proxies understand HTTP, gRPC, WebSocket, and other application protocols. They can route based on URL path, headers, host, or method. NGINX and Envoy are common L7 choices.

If you only need to spread TCP connections across a pool of database replicas, L4 is enough. If you need to send /api/* to one backend and /static/* to another, you need L7.

Choosing between NGINX, HAProxy, and Envoy

NGINX is a good default for web traffic. It is easy to configure, handles static files well, and has a large module ecosystem. It works well as a reverse proxy and an L7 load balancer.

HAProxy is purpose-built for load balancing. Its health checks, stickiness, and connection draining are mature. If you need granular control over TCP and HTTP traffic, HAProxy often behaves more predictably under load than NGINX.

Envoy is built for cloud-native and service mesh environments. It supports dynamic configuration, gRPC, advanced retries, and outlier detection. The trade-off is operational complexity. You usually reach for Envoy when you are already running Kubernetes or a service mesh, not as a simple frontend for a single monolith.

Production patterns worth implementing

TLS termination

Terminate TLS at the proxy so your backend services can speak plain HTTP internally, or use mTLS if you need service-to-service encryption. Do not terminate TLS at the proxy and then re-encrypt to the backend unless you have a compliance reason; it adds latency and certificate management overhead.

Health checks

Configure active health checks so the proxy stops sending traffic to failing backends. Passive health checks—marking a backend down after repeated failures—are also important. Always combine both.

Timeouts

Set explicit timeouts at every hop: client to proxy, proxy to backend, and backend to application. Missing timeouts cause cascading failures where a slow backend exhausts all proxy connections.

Rate limiting

Use the proxy to limit requests per IP or per API key before traffic reaches your application. This protects your backend from abusive clients and accidental traffic spikes.

Sticky sessions

Prefer stateless backends so you do not need sticky sessions. If you must use them for legacy applications with local sessions, configure cookie-based stickiness and plan to remove it.

Debugging common failures

502 Bad Gateway

This means the proxy could not reach the backend. Check:

• Is the backend process running?
• Is it listening on the expected interface, such as 0.0.0.0 versus 127.0.0.1?
• Are firewall rules or security groups blocking the proxy’s subnet?

503 Service Unavailable

Usually means all backends are marked unhealthy or the proxy’s connection queue is full. Review health check logs and backend error rates.

504 Gateway Timeout

The proxy connected to the backend, but the backend did not respond in time. Increase the proxy timeout temporarily, then investigate why the backend is slow. Do not just raise the timeout and forget it; you are hiding the problem.

Header size errors

NGINX returns 400 or 413 when request headers are too large. This often happens with large cookies or SSO tokens. Increase large_client_header_buffers if needed.

WebSocket disconnects

Proxies may close idle WebSocket connections unless you configure hop-by-hop headers such as Connection and Upgrade, and set appropriate read and write timeouts.

NGINX configuration example

http {
    upstream app_backend {
        least_conn;
        server 10.0.1.10:8080 max_fails=3 fail_timeout=30s;
        server 10.0.1.11:8080 max_fails=3 fail_timeout=30s;
        keepalive 32;
    }

    server {
        listen 443 ssl;
        server_name api.example.com;

        ssl_certificate /etc/letsencrypt/live/api.example.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/api.example.com/privkey.pem;

        location / {
            proxy_pass http://app_backend;
            proxy_http_version 1.1;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            proxy_connect_timeout 5s;
            proxy_send_timeout 30s;
            proxy_read_timeout 30s;
        }
    }
}

HAProxy configuration example

global
    log /dev/log local0
    maxconn 2000

defaults
    log global
    mode http
    option httplog
    option dontlognull
    timeout connect 5s
    timeout client 30s
    timeout server 30s

frontend http_front
    bind *:80
    default_backend app_backend

frontend https_front
    bind *:443 ssl crt /etc/ssl/example.com.pem
    default_backend app_backend

backend app_backend
    balance leastconn
    option httpchk GET /healthz
    http-check expect status 200
    server app1 10.0.1.10:8080 check maxconn 100
    server app2 10.0.1.11:8080 check maxconn 100

Production rollout checklist

• TLS certificates are in place and auto-renewal is configured.
• Health check path returns 200 only when the service is truly ready.
• Timeouts are set on client, proxy, and backend.
• Rate limits are configured per client or IP.
• Access logs are shipped to central logging.
• Proxy metrics such as connections, errors, and latency are scraped by monitoring.
• Rolling restart procedure is documented so config changes do not drop active connections.
• Failover behavior is tested by stopping one backend and confirming traffic shifts.

One practical takeaway

Use a reverse proxy to centralize TLS termination, timeouts, and health checks so your application code stays focused on business logic. Start with NGINX or HAProxy, monitor the proxy itself as a critical dependency, and only add Envoy when you need dynamic routing or service mesh features.

IFLEON can help you design and operate the traffic layer for your production infrastructure.