Docker, boot2docker and DNS resolution of containers

... or how to setup a full development environment with docker (on Mac OS X)

This tutorial should help to setup a development environment on Mac OS X that relies heavily on docker and boot2docker. Based on my experience, docker is an excellent tool to create and run applications very effectively during dev, test and production. However, not having great development environment where docker can be used effectively slows you down and requires a lot of hand-tuning each an every time during the development process.

Goal of this tutorial

The goal is to have the following capabilities available after you complete this guide.

Run docker containers from your dev machine and have seamless access to all container using their IPs.
Enable DNS capabilities for all containers and being able to resolve them from your dev machine.
Ensure that this will also work if you have to use a corporate VPN client (like Cisco AnyConnect).
Being able to quickly set this up and tear it down with a simple script.

Prerequisites

Mac OS X (this guide is not required for Linux because it can run docker natively)
boot2docker v1.1.1
docker v1.1.1
git for checking out the git repo with the scripts

What is the current problem?

If you start a docker container on MacOS, it will actually be running inside a VirtualBox VM that boot2docker is setting up for you under the cover. The consequence is that each docker container cannot be pinged or resolved from the Mac which makes it hard to use during development.

Start a docker container and run a server with nc

$ docker run -i -t ubuntu bash
root@4cb806d8ddab:/# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr be:70:6a:45:22:ee
          inet addr:172.17.0.4  Bcast:0.0.0.0  Mask:255.255.0.0
root@4cb806d8ddab:/# nc -l 7777

Try to connect to the server

$ telnet 172.17.0.4 7777
Trying 172.17.0.4...
telnet: connect to address 172.17.0.4: Operation timed out

It will timeout because the IP cannot be resolved. So let's fix that...

Setting up boot2docker

Initialize the boot2docker-vm

$ boot2docker init --dhcp=false --hostip=172.16.0.1

Start boot2docker-vm

$ boot2docker up

Configure host-only network adapter

$ boot2docker ssh "sudo ifconfig eth1 172.16.0.11 netmask 255.255.0.0"

Setting up a route

$ sudo route -n add 172.17.0.0/16 172.16.0.11

This will ensure that all containers which get a 172.17/16 address can be resolved via the host-only adapter.

Setup DNS for Docker

Now that we are able to ping docker containers seamlessly from our Mac, it would be great to have DNS names automatically registered whenever a docker container comes up. SkyDock and SkyDNS fill this gap.

Changing the arguments of the docker daemon

$ boot2docker ssh
docker@boot2docker:~$ sudo pkill bin/docker
docker@boot2docker:~$ sudo /usr/local/bin/docker -d -g /var/lib/docker \
  -H unix:// -H tcp://0.0.0.0:2375 \
  --bip=172.17.42.1/16 --dns=172.17.42.1 &

Launching SkyDNS and SkyDock

$ docker run -d -p 172.17.42.1:53:53/udp --name skydns \
  crosbymichael/skydns -nameserver 8.8.8.8:53 -domain docker
$ docker run -d -v /var/run/docker.sock:/docker.sock --name skydock \
  crosbymichael/skydock -ttl 30 -environment dev \
  -s /docker.sock -domain docker -name skydns

Testing the DNS server

$ docker run -i -t --name u1 ubuntu bash
$ dig @172.17.42.1 +short u1.ubuntu.dev.docker
172.17.0.8

Connecting to VPN will break things

If you have to use Cisco VPN to connect to your corporate network, you will soon realize that once you do that, pinging the docker containers will no longer work. The fix:

$ sudo ipfw -a list | grep "deny ip from any to any"
00411       94       10039 deny ip from any to any
$ sudo ipfw delete 00411

Scripts

I provided two scripts in my git repo. The enable-docker-dns.sh implements the whole procedure described in this blog. The vpn-fix.sh script can be run every time you reconnect to your corporate VPN.