Distcc allows you to distribute the compilation of C, C++ and Obj-C. It wraps your typical compiler executable and offloads the compilation to remote and local compiler instances with the help of the distcc service. This reduces the compile-time of projects with many compilation units.

For distcc to work properly, it should run the same compiler version on all compilation servers. This is typically not a big issue when running the same Linux distribution on both client and servers. But what if your use Arch while the server runs CentOS, or you run Ubuntu Bionic while the server runs Eoan? Or what if you have upgraded your local compiler while the server is still on the previous version of a compiler?

Yes, containers. A Dockerfile, based on Ubuntu Bionic, that installs distcc and gcc-7, 8, 9 and clang-7, 8, 9 with the distcc service as entrypoint looks like this:

FROM ubuntu:18.04

# ENV OPTIONS --allow 1.1.1.1 --allow 2.2.2.2

# Prerequisites for adding the repositories
# Remove the apt cache to keep layer-size small.
RUN apt-get update && \
  apt-get install -y gpg wget && \
  rm -rf /var/lib/apt/lists/*

# GCC
RUN echo "deb http://ppa.launchpad.net/jonathonf/gcc-9.2/ubuntu bionic main" | tee -a /etc/apt/sources.list.d/gcc.list
RUN echo "deb-src http://ppa.launchpad.net/jonathonf/gcc-9.2/ubuntu bionic main" | tee -a /etc/apt/sources.list.d/gcc.list
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 4AB0F789CBA31744CC7DA76A8CF63AD3F06FC659 

# LLVM
RUN echo "deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-9 main" | tee -a /etc/apt/sources.list.d/clang.list
RUN echo "deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-9 main" | tee -a /etc/apt/sources.list.d/clang.list
RUN echo "deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-8 main" | tee -a /etc/apt/sources.list.d/clang.list
RUN echo "deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic-8 main" | tee -a /etc/apt/sources.list.d/clang.list
RUN echo "deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic main" | tee -a /etc/apt/sources.list.d/clang.list
RUN echo "deb-src http://apt.llvm.org/bionic/ llvm-toolchain-bionic main" | tee -a /etc/apt/sources.list.d/clang.list
RUN wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add -

# Install the compilers
# Remove caches to keep layer size small
RUN apt-get update && \
  apt-get install -y \
    distcc \
    gcc gcc-8 gcc-9 \
    g++ g++-8 g++-9 \
    clang clang-8 clang-9 && \
  rm -rf /var/lib/apt/lists/*

# This is the operations port
EXPOSE 3632
# This is the statistics port
EXPOSE 3633

USER distccd

ENTRYPOINT /usr/bin/distccd --no-detach --daemon --stats --log-level info --log-stderr $OPTIONS

That looks promising. Try to build this container and tag it. docker build -t distcc .. This will take a few minutes and leave you with a built and tagged image distcc. A container that runs this image needs to have the OPTIONS environment variable set with the appropriate --allow flags and the necessary ports exposed. We choose to name the container rather unoriginally ‘distcc’.

docker create -p3632:3632 -p3633:3633 -eOPTIONS="--allow 10.12.100.10 --allow 10.8.0.0/24" --name distcc distcc

Now, we could go ahead and start the container and have Docker manage its behaviour. That’s fine.

Another approach for operating systems that use systemd is to configure a service-file to have systemd manage the container. This is an example of /etc/systemd/system/distcc.service.

[Unit]
Description=Dockerized DistCC Worker
Requires=docker.service
After=docker.service

[Service]
Restart=always
ExecStart=/usr/bin/docker start -a distcc
ExecStop=/usr/bin/docker stop -t 10 distcc

[Install]
WantedBy=local.target

Now… systemctl enable distcc.service, and, systemctl start distcc.service.

On the client, configure ~/.distcc/hosts and you are good to go.