[Essay of sorts] Commentary on Pi Image Generation

[faddat]

Hey so for some time I have been working on raspberry pi image generation.

Probably what I end up doing with pi-builder looks more like a fork or use of your design pattern.

I guess that one of the things that I'm looking for here is criticism of my proposed architecture because obviously you have spent a lot of time on this as well.

So far I have been using packer-builder-arm. Everything that would happen inside the darker container here happens inside a chroot over there.

The chroot doesn't allow for some things, like systemctl enable, and if you're using ubuntu, apt upgrade, If that upgrade will trigger kernel changes.

With pi-builder, what I found is that there's probably too many options although I think that this is the closest to ideal solution that I found by a wide margin. So what I'm going to do is I am going to take all of the Dockerfile.part files and make them into one Dockerfile. There are a number of reasons for this but probably the biggest one is that this will be fed into a code generator and spent out the other side and I think it's going to end up being a lot easier to manage as a single Dockerfile, single Makefile and a folder of dependencies.

I could be wrong and that's part of why am making this issue actually. I think that this is a really badly needed product. Currently it's much too hard to create images for embedded devices. It's that difficulty that actually drove me to try and bundle this into starport, our blockchain scaffolding tool.

I might break this out into an independent repository as kind of like a framework similar to pi builder but with fewer options and only in a 64-bit mode.

Ideally, I'd like my tool to be totally separate from the app that runs on top of it.

I just wanted to say thanks so much for your work, there are a lot of things in here that I had been having a difficult time figuring out and I'm just delighted that you made this open source and allowed me to learn from it. I'll link you to what I come up with later tonight. So far it's just been some basic customization and toying around but I think that in the next eight hours or so I'll actually have my setup in a pretty decent spot, owing in large part to your work.

Cheers! -Jacob

[faddat]

Okay! So I did come out with a result.

https://GitHub.com/faddat/spos

I have builds working both in Docker and in a chroot.

neither currently works in GitHub actions but I believe that is a solvable problem. Just running out of space in the VM.

It does use Docker buildx, so there is no longer any need to copy the QEMU static binary into the file system and then remove it at the end.

I will probablyput together a reduced-sized version as well, this initial one is designed for use by developers and contains a wide suite of development tools.

Once again thank you all so much for your help.

[mdevaev]

Thank you for these words :)

[faddat]

Hey so I thought I would keep the essay going. All right, I am a little more than a month in. I have successfully done all of this with docker buildx, it's great.

I definitely prefer this approach to using packer, which is what I was doing before. In general, builds are faster, And the whole thing works quite well with github actions.

I have stuck to a quite linear approach. So basically, each device has its own Docker file and that darker file builds it from end to end. Interestingly, the darker images that I am Producing are usually consumed by other build systems, like:

https://github.com/cosmos/gaia https://github.com/tendermint/spn https://github.com/tendermint/starport https://gitlab.com/blurt/blurt https://github.com/virtualeconomy/v-systems

There will probably be more soon, I am attempting to make this a widely used pattern for providing cheap nodes for fast proof of stake blockchains.

In the case that you had an application that you wanted to add across the number of devices it might be that the best way to do this is actually by using a matrix build, building from all of the device specific images created by SOS in CI, And then adding your application binaries and configuration as the last step across many images.

The decision to go with arch and do all this work to support it really seems to have paid off. Blurt is kind of a behemoth and I was actually pretty surprised that I was able to get it onto a pi, and I give a lot of credit to the efficiency and fresh package is that one can find in arch.

Probably the only downside with arch is the size of the rootFS.

When decompressed I think it's over 450 MB. I have thought about using the same concept of matrix builds to also support alpine Linux but I don't want to expand the scope too much and I am very happy with the way that arch has performed so far. For now, I think it is worth dealing with the larger root file system.

Here's my big ol issue:

https://github.com/tendermint/starport/issues/401