Building for armel on Linux x64 using Docker

Daniel F. Dickinson - 25 November, 2019

Tags:

What is in this article

Overview

The cross-compilation toolchains builtin to most modern Linux distributions do not support older versions of GCC. For old kernels (and other software) that require GCC4 or lower for building, this poses a challenge. One either needs to build a cross-compilation toolchain or use virtualization. This article describes setting up a virtual ARM environment for doing armel (ARMv5) compilation using docker containers.

On Debian Buster and versions of Ubuntu based on the same vintage qemu-user, this technique is not useful due to a bug: https://bugs.launchpad.net/qemu/+bug/1805913.

Even without this bug, a better technique is to use build a virtual machine and to run whatever scripts or builds one wants in the VM, as described in Cross-Compile for Armel Using an ARM HF VM.

Nonetheless if you still want to try the (slower) Docker technique, it is described below.

Preliminaries

We’re going to use Debian’s armel port for this exercise.
We’re using Debian Jessie for GCC4 support. If you need older GCC, then you need an appropriate OS, or to build a cross-compilation toolchain.
The instructions are written assuming we are running on a Debian system. However, debootstrap is also available for use on non-Debian systems as is an up-to-date docker.io.
You need debootstrap and docker on your system (apt-get install debootstrap docker.io)
If you have access to a prebuilt Debian armel rootfs you can skip directly to creating the container.
This article assumes you have Docker configured and running

Create the Rootfs

Use the script to create a Debian Jessie rootfs below.

What the Script Does

Create the Base System

sudo apt-get install qemu-user-static
- On Debian 10 this will automatically register ARM EABI binaries for execution by qemu-arm-static (userspace emulation of the ARM architecture on non-ARM architectures such as x86).
- For older distributions the registration may not be automatic, in which case you need to look up binfmt registration, e.g. see ~~https://ownyourbits.com/2018/06/13/transparently-running-binaries-from-any-architecture-in-linux-with-qemu-and-binfmt_misc/~~.
mkdir -p armel-root/usr/bin
cp /usr/bin/qemu-arm-static armel-root/usr/bin/qemu-arm-static
sudo debootstrap --arch=armel --foreign jessie "$(pwd)"/armel-root
cp /usr/bin/qemu-arm-static armel-root/usr/bin
sudo PATH=/bin:/sbin:/usr/bin:/usr/sbin /sbin/chroot armel-root /usr/bin/qemu-arm-static /bin/bash
export LANG=C.UTF-8
Depending on what terminal you are using, you may need to also do: export TERM=xterm-color. (This is only required if your terminal is not supported by a default Debian Jessie install).
/debootstrap/debootstrap --second-stage

Prepare the Base System for First Boot

Assuming you are still in the chroot:

Follow the rest of the instructions at https://www.debian.org/releases/jessie/armel/apds03.html.en#idp51134208 (more details below).

mount -t proc proc /proc
apt-get install makedev
cd /dev
MAKEDEV generic

Create /etc/fstab, for example (for virtualization):

# /etc/fstab: static file system information.
#
# file system mount point type options dump pass
/dev/sda / ext4 defaults 0 1
proc /proc proc defaults 0 0

Create /etc/network/interfaces, for example:

######################################################################
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
# See the interfaces(5) manpage for information on what options are
# available.
######################################################################

# The loopback interface isn't really required any longer, but can be used
# if needed.
#
auto lo
iface lo inet loopback

# To use dhcp:
#
auto eth0
iface eth0 inet dhcp

# An example static IP setup: (network, broadcast and gateway are optional)
#
# auto eth0
# iface eth0 inet static
# address 192.168.0.42
# network 192.168.0.0
# netmask 255.255.255.0
# broadcast 192.168.0.255
# gateway 192.168.0.1

# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d

apt-get install resolvconf
echo jessie-armel-build > /etc/hostname

Update /etc/apt/sources.list. For example:

deb http://deb.debian.org/debian jessie main contrib non-free
deb-src http://deb.debian.org/debian jessie main contrib non-free
deb http://deb.debian.org/debian-security jessie/updates main contrib non-free
deb-src http://deb.debian.org/debian-security jessie/updates main contrib non-free
deb http://deb.debian.org/debian jessie-updates main contrib non-free
deb-src http://deb.debian.org/debian jessie-updates main contrib non-free

apt-get update && apt-get upgrade
apt-get install locales && dpkg-reconfigure locales
apt-get install console-setup && dpkg-reconfigure keyboard-configuration
apt-get install sudo
adduser builder
adduser builder sudo
tasksel install standard
apt-get clean
/etc/init.d/exim4 stop
/etc/init.d/cron stop
/etc/init.d/rsyslog stop
/etc/init.d/dbus stop
pkill -TERM rsyslogd
umount /proc
exit

Create a Tarball of the Rootfs

tar -C armel-root --remove-files --one-file-system -czf jessie-armel-build.tar.gz .

Using a Script

NB: The script is interactive and you will be prompted at some points. If you’re looking for complete automation you will need to modify the script appropriately.

Copy the script below to create-jessie-armel-build.sh
sudo sh -c create-jessie-armel-build.sh
sudo chown *yourusername*:*yourusername* jessie-armel-build.tar.gz

#!/bin/sh
set -e

apt-get install qemu-user-static
mkdir -p armel-root/root
mkdir -p armel-root/usr/bin
cp /usr/bin/qemu-arm-static armel-root/usr/bin/
debootstrap --arch=armel --foreign jessie "$(pwd)"/armel-root

cat >armel-root/root/prepare-debian-rootfs.sh <<EOF
#!/bin/sh

set -e

export LANG=C.UTF-8
export TERM=xterm-color
/debootstrap/debootstrap --second-stage
mount -t proc proc /proc
cd /dev
apt-get -y install makedev
MAKEDEV generic

cat >/etc/fstab <<FSTABEOF
# /etc/fstab: static file system information.
#
# file system mount point type options dump pass
/dev/sda / ext4 defaults 0 1
proc /proc proc defaults 0 0
FSTABEOF

cat >/etc/network/interfaces <<IFACEEOF
######################################################################
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
# See the interfaces(5) manpage for information on what options are
# available.
######################################################################

# The loopback interface is not really required any longer, but can be used
# if needed.
#
auto lo
iface lo inet loopback

# To use dhcp:
#
auto eth0
iface eth0 inet dhcp

# An example static IP setup: (network, broadcast and gateway are optional)
#
# auto eth0
# iface eth0 inet static
# address 192.168.0.42
# network 192.168.0.0
# netmask 255.255.255.0
# broadcast 192.168.0.255
# gateway 192.168.0.1

# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d
IFACEEOF

apt-get -y install resolvconf
echo jessie-armel-build >/etc/hostname

cat >/etc/apt/sources.list <<SRCEOF
deb http://deb.debian.org/debian jessie main contrib non-free
deb-src http://deb.debian.org/debian jessie main contrib non-free
deb http://deb.debian.org/debian-security jessie/updates main contrib non-free
deb-src http://deb.debian.org/debian-security jessie/updates main contrib non-free
deb http://deb.debian.org/debian jessie-updates main contrib non-free
deb-src http://deb.debian.org/debian jessie-updates main contrib non-free
SRCEOF

apt-get update && apt-get -y upgrade
apt-get -y install locales && dpkg-reconfigure locales
apt-get -y install console-setup && dpkg-reconfigure keyboard-configuration
apt-get -y install sudo
adduser builder
adduser builder sudo
tasksel install standard
apt-get clean
/etc/init.d/exim4 stop || true
/etc/init.d/atd stop || true
/etc/init.d/rsyslog stop || true
/etc/init.d/cron stop || true
/etc/init.d/dbus stop || true
sleep 2
pkill -TERM rsyslogd || true
sleep 2
/etc/init.d/cron stop || true
sleep 2
umount /proc
EOF

chmod 755 armel-root/root/prepare-debian-rootfs.sh
cp /usr/bin/qemu-arm-static armel-root/usr/bin
PATH=/bin:/sbin:/usr/bin:/usr/sbin /sbin/chroot armel-root /usr/bin/qemu-arm-static /bin/bash -c /root/prepare-debian-rootfs.sh || {
echo "Failed to chroot; cant create rootfs!"
exit 1
}
echo "Created rootfs; creating tarball from rootfs"

tar -C armel-root --remove-files --one-file-system -czf jessie-armel-build.tar.gz .

rm -rf armel-root

echo "Done. Created tarball of rootfs."

Create the Docker Image

Create the Base Image

cat jessie-armel-build.tar.gz | docker import - jessie-armel:latest

Build the Development Environment Container

Create a directory (e.g. mkdir workdir) and change to it (e.g. cd workdir).

Create a file called Dockerfile such as:

FROM jessie-armel:latest

RUN apt-get update && \
apt-get -y upgrade && \
apt-get install -y \
bash-completion \
binutils-dev \
build-essential \
bzip2 \
cmake \
curl \
debhelper \
dh-systemd \
dpkg-dev \
diffutils \
fakeroot \
fakechroot \
git \
gnupg2 \
libbfb0-dev \
libc6-dev \
libcurl4-gnutls-dev \
libdw-dev \
libelf-dev \
ncurses-dev \
pkg-config \
python-minimal \
quilt \
u-boot-tools \
wget \
xmlto \
xz-utils \
zlib1g-dev

RUN apt-get -y build-dep linux-source && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

RUN mkdir -p /home/builder/Build && \
chown builder:builder /home/builder/Build

VOLUME ["/home/builder/Build"]

USER builder

WORKDIR /home/builder/Build

CMD ["bash"]

In the directory with the Docker file, execute
```
docker -t jessie-armel-build:latest build .
```

Using the container

If you need additional packages or software you can install them by using another Dockerfile with FROM jessie-armel-build:latest and the appropriate Dockerfile commands for what you want to add.
You may want to add a user with user and group id that matches the user that owns /your/build/directory (below) and change the USER user in the above Dockerfile (or your extending Dockerfile), in order to avoid permissions issues.
When you want an interactive shell that uses the software in the container, you can do: docker run -it -v /your/build/directory:/home/builder/Build jessie-armel-build:latest bash