arm/arm64 images and no wallet

[meeDamian]

Hey guys, truly amazing work!

Two questions however: have you considered also building and uploading arm[64] images (alpine images would be great to have available on Raspberry Pi-class computers).

Additionally, it would be very handy to have a version of the image built w/o the wallet - useful for nodes that only function as the source of truth, ex a backend for Lightning Network node.

[meeDamian]

Just a quick update on the topic of arm images:

After a small adjustment (make -j4 -> make) alpine images build correctly on both Raspberry Pi 3 as well as 0 (Note: zero needed a bit more SWAP than the 100MB enabled by default).

That being said, alpine does not have a separate base image for arm32v7 architecture, and arm32v6 is used on both instead, therefore only one arm image covers pretty much the entire Raspberry Pi family (as well as many of their clone projects).

[victorperin]

Maybe we can do that building images on travis using some emulation process.

I don't know much about it. Buy I'm reading some posts like that: https://www.tomaz.me/2013/12/02/running-travis-ci-tests-on-arm.html

[meeDamian]

I'm no expert myself, but for some reason, my amd64 Mac is able to run images I've build on a Raspberry Pi 3 with: docker run --rm -it meedamian/bitcoind:linux-arm32v7-latest -printtoconsole, so there must be some emulation happening.

Emulation also got confirmed by the author of this article in a tweet here. And there, he seemingly builds an arm image on an amd64 CPU, by simply building it from a file called Dockerfile.arm:

docker build --file ./Dockerfile.arm --tag ${IMG_NAME}:linux-arm-latest .

edit. I'm mentioning the article above, also because it makes dealing with multiple CPU architectures a lot nicer.

[meeDamian]

Hey, yet another update :).

I've figured out how to build arm images on travis here, however:

1. Seemingly the only way to do it is by using qemu, which is really slow and travis builds keep hitting against the 50 minutes execution limit and failing,
2. I needed to modify the Dockerfile slightly further (see lines 4, 31, and 96), arm builds need it to COPY a previously downloaded matching version of qemu, while amd64 ones just copy a dummy file for now,
3. AFAICT you're not using Docker manifests yet, switching to supporting multiple architectures in a sane way would require that, and I do that here.

Given all that, I was wondering if it didn't make sense to split the Dockerfile stages into separate Dockerfiles (full disclosure: I have no idea if that's even possible yet…), after all the berkeleydb used across all bitcoin versions is exactly the same, and it doesn't necessarily make sense to keep rebuilding it with each new bitcoin release(?). Alternatively arm images could be built completely w/o wallet, but that seems somewhat confusing and unnatural…

Knowing all that, please let me know if you'd be interested merging a PR adding arm support, if I manage to get it working (at least for the alpine images). Personally I'd much rather contribute to your excellent effort than, forked and created a yet another Docker image for bitcoin 😅.

[meeDamian]

Okay, even after splitting berkeleydb stage into a separate image, and re-enabling make -j4 (while it crashes on devices, it works fine of Travis), the build still [times out on the 50 minute mark]…

I could probably also extract building the base machine for the bitcoind build out of it, and just rebuild that base periodically (once a week?), but I'm pretty sure that it won't suffice either, and it probably makes more sense to entertain other options instead…

One option that comes to my mind, and that I've verified to work is this thing, however I'd be rather reluctant trusting them with my dockerhub creds (for automated pushing), and it seems they might've ceased working on it (cc. @metakermit) :/.

Other options include:

1. finding another CI provider that does not impose a 50 mins execution limit,
2. building & uploading images manually, although that adds an extra overhead and introduces a little bit more trust into the system,
3. building images directly on arm devices, this would probably require a home-baked CI system on RBPs, but would allow for the removal of qemu. It, just like the previous option, introduces a little bit more trust into the system.

[meeDamian]

First of all, forgive my updates spam, hope you don't mind it 😐.

Other than that, I've looked into both @circleci and @codeship and the first one doesn't seem to be able to register qemu at all (doesn't have kernel support and prohibits privileged calls), while it seems theoretically possible on the latter it would need to be run on Codeship Pro (free 100 builds/mo), and in a Docker within a Docker emulating architecture of another CPU, so… even though they have a 3 hours timeout per build, it seems a bit unlikely to work as well…

Open to suggestions by anyone with a bit more experience in Docker, qemu and arm than my week-long adventure 😅.

[metakermit]

One option that comes to my mind, and that I've verified to work is this thing, however I'd be rather reluctant trusting them with my dockerhub creds (for automated pushing), and it seems they might've ceased working on it (cc. @metakermit) :/.

CC @doublemalt

[meeDamian]

Tried another thing based on this article and while it did allow me to build an arm image on the Docker Cloud it's constrained to a single core only, and timed out after 4 hours of running. Also, it is a very hacky way, and I'm not sure if I'd be comfortable running Bitcoin Node built this way.

[ruimarinho]

@meeDamian what you're looking at is something similar to INCLUDE which is not supported by Docker. The only thing we can leverage are same-file multi-stage builds. It is not necessary to use qemu for building a variant for ARM, provided you have the right toolchain (only works in Ubuntu).

In order to support the "no wallet" variant, I would have to setup a templating system to enable other variants (no mqtt, no upnp, etc).

[meeDamian]

Hey, thank you for thoughtful response!

what you're looking at is something similar to INCLUDE which is not supported by Docker.

I guess something similar, as long as it's just file-based, can be achieved with:

COPY --from=<image-name>:<tag>  <path-in-image>  <local-path>

Example:

# Build stage for Bitcoin Core
FROM arm32v7/alpine:3.10 AS bitcoin-core

# fetch already built berkeleydb
COPY --from=lncm/berkeleydb:db-4.8.30.NC /opt /opt

The only thing we can leverage are same-file multi-stage builds.

Just a note that now with the intro of DOCKER_BUILDKIT=1 it's possible to build to a specific stage, by doing sth like:

DOCKER_BUILDKIT=1 docker build --target builder .

It is not necessary to use qemu for building a variant for ARM, provided you have the right toolchain (only works in Ubuntu).

I know, I did try building my own toolchain for Alpine and musl. Unfortunately, after a few miserable days of trying I've given up, and decided to go with qemu, which seems easier. For qemu, the only extra steps necessary are:

1. Register qemu in kernel by running something like below on the host device:

   docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

2. Make sure that arch-specific images are used in the FROM directive, example:

   FROM arm32v6/alpine:3.10
   
   # instead of
   
   FROM alpine:3.10

In order to support the "no wallet" variant, I would have to setup a templating system to enable other variants (no mqtt, no upnp, etc).

Right. That would explode the number of images generated. So I reckon it might be better to just leave things as they are.

---

BTW: Now I'm experimenting with Github Actions, as their timeout per-job is 6 hours, and I did manage to get Bitcoind compiled for arm, using qemu here: https://github.com/meeDamian/docker-bitcoind/runs/197832660

Note it took 5h 59m 22s out of allowed 6h 😂.

[olljanat]

@meeDamian there is no need to use emulator on this one. Dockerfile just need to be structured on way that there is no need to use RUN instruction on ARM. I added PR https://github.com/ruimarinho/docker-bitcoin-core/pull/90 which does that for 0.19 version on Debian based image.

[ruimarinho]

Closed by https://github.com/ruimarinho/docker-bitcoin-core/pull/93. (I might address the nowallet request later on).