Support more architectures

[Noki]

Currently the docker image is build for linux/amd64. It would really help to have it build for more architectures e.g. linux/arm (raspberry pi 2) or linux/arm64 (raspberry pi 3/4).

[ro-tex]

Hello @Noki,

That makes sense, especially since we provide binaries for Raspberry Pi.

It seems like the best approach is to use arm32v7/alpine as base image. Their supported architectures include amd64, arm32v6, arm32v7, arm64v8 which should cover all Pi models.

I have limited experience with Pi, especially with the later models. If you have any concerns about the chosen base image or if you have any tips on useful parameters for the Pi that need to be built into the image - please let me know.

[Noki]

@ro-tex Alpine images are great, however arm32v7/alpine is just the alpine image for the arm32v7 architecture. As far as I know you have to build an image for each architecture and combine them using a manifest file. There are plenty of guides out there, however I've never done it on Docker Hub.

Here is an docker multi arch example: https://github.com/ckulka/docker-multi-arch-example

You can see that there are multiple arch specific docker files and a manifest combining them.

Hope this helps.

[ro-tex]

A multi-arch image sounds like a good idea but it has some drawbacks. One of them is that the complexity seems to outweigh the benefits of having such an image.

Another issue is that running siad on Raspberry Pi 1/2/Zero is problematic. Those devices are underpowered and have very limited amount of RAM. Providing an official image is not exactly promising support but it's close enough for people to open bugs which we will need to triage and debug before we can definitively say that they are caused by the limited resources of the system. At this point this doesn't seem like the best use of the limited developer time we have.

That said, building siad for ARMv6 and running it on a Raspberry Pi 1 is possible. I did it today on my old Pi 1B in order to assess the feasibility of the setup. (I waited for more than half an hour for siad to start up and begin syncing the blockchain and I gave up before it managed to succeed.) Given that we have the dev image which provides the "get the source and build it" step and the new pi image which provides the "pack it in an ARM container" step, writing an ARMv6 image is trivial, should anyone really need it. (Just replace make release with GOOS=linux GOARCH=arm GOARM=6 make release and you should be set.)

I've opened an MR (#18) which introduces an ARMv8 image which covers Pi 3 and 4. This is in alignment with the binaries we provide - we only provide pre-compiled binaries for ARMv8.

[Noki]

Multi-arch docker images reduce the complexety of documentation, setup and orchestration, and is less error prone than providing individual images. This helps people to get started and reduces frustration. Even if the setup to build the images is slightly more complex, it's usually worth the effort to onboard more users.

Regarding the hardware of the Raspberry Pi 1/2/Zero you are absolutely right, however there are quite some older alternatives out there based on arm32v6 and arm32v7 that are way more powerful than the old raspberry pi's (some NAS systems and quite some of this list). In my opinion it is better to support the architectures and give recommendations regarding minimal hardware requirements.

Instead of packaging the pre compiled binaries into the docker images you could also just build them from source directly on docker hub.

[ro-tex]

This makes sense. I see value in providing a readily available image to pretty much everyone running an ARM-based machine. The downside is that this will take some time and right now there are higher priority items on the agenda.

I'll do this:

1. I'll keep this ticket open, so it doesn't fall through the cracks and it gets followed up on when time allows.
2. I'll publish the already made ARMv8 image as an interim solution until enough time is available for the fully-fledged one.

Thanks for the pointer to the guide for building multi-arch images, this will be my starting point. If you happen across another/better - please share. :)

[n8maninger]

If you're compiling Sia from source you can use docker buildx --platforms linux/amd64,linux/arm64,linux/arm/v6,linux/arm/v7 -t nebulouslabs/sia:latest --push to build and push the platforms all at once without having to mess with the manifest manually.

[ro-tex]

I finally managed to get a Raspberry Pi 4 and a Raspberry Pi 1B and found some time to play with them.

Unfortunately, what I found out is not that good - Sia needs a 64bit system in order to run. The 64bit ARM is the aarch64 architecture, supported by Pi 3 and 4. The reason for this is that boltdb, which we use for storing and processing the consensus, uses memory mapped I/O, meaning that it's capped at a little under 4GB while our consensus set is north of 20GB at this point. So, you can compile siad and run it and it will start without a problem but once it syncs the first 4GB of the blockchain it will die with either panic: runtime error: index out of range [21476556800] with length 2147483647 or file too small. I experienced the first one on the Pi 4 with a 32bit Raspberry Pi OS (armv7l) and the second on the Pi 1B (armv6l). Running siad on the Pi 4 with a 64bit Raspberry Pi (aarch64) OS is not a problem at all.

This means that while we have the 64bit limitation set by boltdb we cannot support 32bit architectures and we already support the only 64bit ARM architecture available.

The good news is that we might have something in mind that could help with that but it's far too early to talk about it because it's (at best) on a whiteboard level.