As a senior sysadmin, I've seen it hundreds of times. You have great hardware, fast NVMe drives, and a 100Gbps network, but your application chokes at 10k concurrent connections. Why? Because Linux kernel defaults are designed for desktops and general-purpose servers from 2010, not for beasts handling production traffic at massive scale.
Network Stack Optimisation
The most critical element under heavy traffic (HTTP/S, gRPC, WebSocket) is how the kernel handles incoming queues and file descriptors. Forget the default somaxconn of 128. That's a performance killer.
# Increase the maximum number of pending connections
net.core.somaxconn = 65535
# Maximum number of packets waiting in the incoming queue
net.core.netdev_max_backlog = 16384
# TCP buffer optimisation (Read/Write)
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
# Fast release of idle connections (TIME_WAIT reuse)
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_max_syn_backlog = 8192
Setting tcp_tw_reuse allows reuse of sockets in the TIME_WAIT state, which is critical when your service rapidly opens and closes thousands of connections per second. Without it, you risk exhausting the ephemeral port pool.
Memory Management and VFS
The next flashpoint is I/O and cache. The kernel loves to swap aggressively if you let it. On a database server or cache layer (Redis/Memcached), you want to avoid this at all costs.
# Minimise swap usage (0-10 for servers)
vm.swappiness = 5
# Percentage of dirty memory at which processes start writing to disk
vm.dirty_ratio = 15
# Percentage of dirty memory at which the kernel starts background writes
vm.dirty_background_ratio = 5
# Aggressiveness of releasing inode cache (default 100)
vm.vfs_cache_pressure = 50Applying Changes
After editing the configuration files, reload the new settings without restarting the server:
# sysctl -p /etc/sysctl.d/99-network-tuning.conf
# sysctl -p /etc/sysctl.d/99-memory-tuning.conf