Emulate a Redis Failover with Docker

Reading the Redis documentation can be a bit confusing without the hands-on experience. You could in theory create multiple processes of the Redis Server on your machine and configure each of them in part, but what if you could do it in a few commands? Not only that but emulate the network they’re connected to as well.

I’ve been looking into this and there’s a few examples out there on Web, the best one I could find was this one: https://github.com/AliyunContainerService/redis-cluster

So, starting from that example, I’ve tried to do the next best thing, which is to create a single docker-compose.yml file for all of it. Removing the need to build each image, just to do a docker-compose up and scale as needed.

Here’s what I got:

master:
  image: redis
slave:
  image: redis
  command: redis-server --slaveof master 6379
  links:
    - master
sentinel:
  image: redis
  command: >
    bash -c "echo 'port 26379' > sentinel.conf &&
    echo 'dir /tmp' >> sentinel.conf &&
    echo 'sentinel monitor master master 6379 2' >> sentinel.conf &&
    echo 'sentinel down-after-milliseconds master 5000' >> sentinel.conf &&
    echo 'sentinel parallel-syncs master 1' >> sentinel.conf &&
    echo 'sentinel failover-timeout master 5000' >> sentinel.conf &&
    cat sentinel.conf &&
    redis-server sentinel.conf --sentinel"
  links:
    - master
    - slave

Basically, after saving this into a docker-compose.yml file and running docker-compose up in that folder you’ll get this:

You can now scale as needed. For example, by running:

docker-compose scale slave=2 sentinel=3

You’ll end up with:

To initiale a failover, you’ll need to take the master out of the picture, you can do that with:

docker-compose pause master

You can now observe the communication between the sentinels and slaves. After the down-after-milliseconds and failover timeout passes, one of the slaves will be selected for promotion.

After the sentinels agree on the selection, the slave will become the new master.

You can now unpause the old master by doing this:

docker-compose unpause master

The old master will now become a slave of the new master and perform a sync.

That’s about it. As an exercise you could try setting up a cluster starting from this and observe failovers there.