How to Use Docker Multi-Stage Builds

Docker multi-stage feature allows to make your dev environment as close to production as possible.

While Docker is an industry standard for deploying web applications to production environments, it also comes in handy for local development.

When you decide to use Docker for both purposes, a few concerns arise :

• for simplicity reasons and to respect the DRY principle, you don’t want to use one Dockerfile per environment, but instead, have a single file for both local development and production. You want the production image to be a copy of what has been used for development (“it works on my machine, therefore it works anywhere”)
• on the other hand, you might need to use different configurations for dev and prod environments, which cannot be easily handled at runtime. Examples of this include: installing some specific packages (a test suite for development, a production web server…), binding the container’s app directory to a local volume in development (to use some features such as live-reload and debug servers), and other things.

Fortunately, Docker has a built-in solution for this, and it is called multi-stage builds.

Docker Multi-Stage with Django

Assume you have a Django web app you want to productionize.
A simple Dockerfile for this app might look like this

FROM python:3.8.3-buster

# copy package information and source code 
COPY requirements.txt /code/
COPY ./app/ /code/

WORKDIR /code

# install packages
RUN apt-get update -y
RUN pip install -r requirements.txt
RUN pip install gunicorn==20.1.0
CMD python manage.py collectstatic --no-input && python manage.py migrate && gunicorn app.wsgi:application -c gunicorn.conf.py --bind 0.0.0.0:8000

Now say we also want to use this Dockerfile for development.
You don’t need to understand Python to see the problem. Basically, some instructions in this Dockerfile will be used only for development, and some only for production.

In development :

• we don’t want to copy the source code inside the container, because we will use our local filesystem, therefore we don’t need COPY ./app/ /code/
• we will need to install some development requirements as well, which come from another file named requirements.dev.txt
• we don’t want to install gunicorn, and the last command can be replaced by python manage.py runserver, which runs a dev server

Let’s see how we can do that with multi-stage builds.

Let’s write it!

FROM python:3.8.3-buster as base
COPY requirements.txt /app/
WORKDIR /app
RUN apt-get update -y
RUN pip install -r requirements.txt

FROM base as dev
COPY requirements.dev.txt /app/
RUN pip install -r requirements.dev.txt
CMD python manage.py runserver

FROM base as prod
COPY ./app/ /app/
RUN pip install gunicorn==20.1.0
CMD python manage.py collectstatic --no-input && python manage.py migrate && gunicorn app.wsgi:application -c gunicorn.conf.py --bind 0.0.0.0:8000

In this example we divided or monolithic Dockerfile into three ‘stages’:

1. The first stage is the ‘base’. It contains everything that is needed in development and production, i.e, installing OS packages and python packages (with requirements.txt).
2. The second stage builds the ‘dev’ image. It takes the result of the base image and installs the dev packages (with requirements.dev.txt). Finally, the Docker command launches a dev server.
3. The last stage in the prod environment. Here we want to copy the source code and install gunicorn, which allows us to productionize Django apps.

Note that both the ‘dev’ and ‘prod’ stages inherit from ‘base’.

Now you can build any image separately :

docker build . -t myImage --target base # stops at 'base' stage
docker build . -t myImage --target dev # builds dev image
docker build . -t myImage --target prod # builds prod image
docker build . -t myImage # also builds prod image as it is the last one

Now you can easily use that in your CI and your local environment.