docker networking performance

While trying to determine the performance overhead of a simple docker container setup, I discovered several issues related to network performance.

two issues I have encountered

  • docker-proxy handles local network traffic when using the hosts bridge IP (or localhost)
  • connection tracking overhead of NAT increases memory usage

my setup

I am using docker containers as an easy way to deploy backends on a host. I wish to use the hosts bridge IP as a backend to route requests from the load balancer. The load balancer is just a haproxy instance running on port 8080 and it uses 500X as backend ports. I had to doctor up the output a little but this is still the output of docker ps.

07 haproxy:0.0.1-8 X Up 2 days   0.0.0.0:443->443/tcp     dev1-haproxy-1      
                                 0.0.0.0:1983->1983/tcp
                                 0.0.0.0:8080->8080/tcp

b6 api:4.2.0-2   X  Up 14 hours  0.0.0.0:5003->8080/tcp   dev1-api-3
f0 api:4.2.0-2   X  Up 14 hours  0.0.0.0:5002->8080/tcp   dev1-api-2
7f api:4.2.0-2   X  Up 15 hours  0.0.0.0:5001->8080/tcp   dev1-api-1

I’ve measured performance of docker-proxy process using nginx and timing the transfers and have seen performance degrades around 40-50% due to docker having to reverse proxy the connection in software.

I really believe that if performance is important, there should be a option of disabling docker-proxy, but I will focus on this more later if I really need to.

issues with docker-proxy

  • software reverse proxy should not be enabled by default
  • network performance degradation up to 50% is not acceptable when traffic is accidently routed through the proxy.

i’m not entirely sure why the userland proxy is even required or why it was introduced. investigating the proxy history shows proxy was introduced in Apr 2013, commit 930e9a7, unfortunately it is not easy to determine the intention and why iptables was not just used in the first place to handle local traffic. iptables is used to route the network traffic externally, so I wonder, why the proxy in userland. It seems pointless but I have yet to investigate it further.