distcc as a Docker system service

Distcc allows you to distribute the compilation of C, C++ and Obj-C. It wraps your typical compiler executable and offloads the compilation to remote and local compiler instances with the help of the distcc service. This reduces the compile-time of projects with many compilation units.

For distcc to work properly, it should run the same compiler version on all compilation servers. This is typically not a big issue when running the same Linux distribution on both client and servers. But what if your use Arch while the server runs CentOS, or you run Ubuntu Bionic while the server runs Eoan? Or what if you have upgraded your local compiler while the server is still on the previous version of a compiler?

Yes, containers.

A Dockerfile, based on Ubuntu Bionic, that installs distcc and gcc-7, 8, 9 and clang-7, 8, 9 with the distcc service as entrypoint looks like this:

FROM ubuntu:18.04

# ENV OPTIONS --allow --allow

# Prerequisites for adding the repositories
# Remove the apt cache to keep layer-size small.
RUN apt-get update && \
  apt-get install -y gpg wget && \
  rm -rf /var/lib/apt/lists/*

RUN echo "deb bionic main" | tee -a /etc/apt/sources.list.d/gcc.list
RUN echo "deb-src bionic main" | tee -a /etc/apt/sources.list.d/gcc.list
RUN apt-key adv --keyserver --recv-keys 4AB0F789CBA31744CC7DA76A8CF63AD3F06FC659 

RUN echo "deb llvm-toolchain-bionic-9 main" | tee -a /etc/apt/sources.list.d/clang.list
RUN echo "deb-src llvm-toolchain-bionic-9 main" | tee -a /etc/apt/sources.list.d/clang.list
RUN echo "deb llvm-toolchain-bionic-8 main" | tee -a /etc/apt/sources.list.d/clang.list
RUN echo "deb-src llvm-toolchain-bionic-8 main" | tee -a /etc/apt/sources.list.d/clang.list
RUN echo "deb llvm-toolchain-bionic main" | tee -a /etc/apt/sources.list.d/clang.list
RUN echo "deb-src llvm-toolchain-bionic main" | tee -a /etc/apt/sources.list.d/clang.list
RUN wget -O - | apt-key add -

# Install the compilers
# Remove caches to keep layer size small
RUN apt-get update && \
  apt-get install -y \
    distcc \
    gcc gcc-8 gcc-9 \
    g++ g++-8 g++-9 \
    clang clang-8 clang-9 && \
  rm -rf /var/lib/apt/lists/*

# This is the operations port
# This is the statistics port

USER distccd

ENTRYPOINT /usr/bin/distccd --no-detach --daemon --stats --log-level info --log-stderr $OPTIONS

That looks promising. Try to build this container and tag it. docker build -t distcc . . This will take a few minutes and leave you with a built and tagged image distcc. A container that runs this image needs to have the OPTIONS environment variable set with the appropriate --allow flags and the necessary ports exposed. We choose to name the container rather unoriginally ‘distcc’.

docker create -p3632:3632 -p3633:3633 -eOPTIONS="--allow --allow" --name distcc distcc

Now, we could go ahead and start the container and have Docker manage its behaviour. That’s fine.

Another approach for operating systems that use systemd is to configure a service-file to have systemd manage the container. This is an example of /etc/systemd/system/distcc.service .

Description=Dockerized DistCC Worker

ExecStart=/usr/bin/docker start -a distcc
ExecStop=/usr/bin/docker stop -t 10 distcc


Now… systemctl enable distcc.service, and, systemctl start distcc.service.

On the client, configure ~/.distcc/hosts and you are good to go.