search

pikvm / pi-builder

Extensible tool to build Arch Linux ARM for Raspberry Pi on x86_64 host using Docker
https://pikvm.org
GNU General Public License v3.0
94 stars 60 forks source link

ARM64 Support #4

Closed faddat closed 9 months ago

faddat commented 3 years ago

Some of our software has known security issues in 32 bit environments.

Would you accept a patch that extends pi-builder to support the creation of 64 bit images?

mdevaev commented 3 years ago

Hi! Yes, why not. Check the commit #6d5ccc76a2bc7b0b829a94ec99150accac53e3df to start. I used to have aarch64 support, but I removed it because I didn't use it. it seems that there was a basic working capacity, but it was a long time ago and I don't remember the details anymore. Note that QEMU 5.1 is broken, so you need to use QEMU 5.0.1 right now.

faddat commented 3 years ago

That's wonderful.

I haven't made the transition yet but my expectation is that I will be able to use your software to dramatically reduce our image build times.

mdevaev commented 3 years ago

Ha-ha, well, that's the main reason I did it :) This helped me a lot when developing Pi-KVM, otherwise I could have waited hours for the build to complete.

faddat commented 3 years ago

So even with us, build times are still kind of an issue for me. I am hoping to figure out a way to make this even faster and possibly slim down arch linux arm. `bout 5 years ago I remember setting up Arch for an Orange Pi, and I recall just using pacstrap. I don't know if it's still possible or what might be making the initial download the 400 MB that it is but I am hoping to reduce that as well.

For the kind of thing I'm working on, less software ends up being exponentially more.

faddat commented 3 years ago

If it's not currently desired then I can tell you that the default raspberry Pi 64-bit kernel is actually not all that stable and you probably don't want to use it so I think that for now I'm going to close this issue.

I solved the problem by using a user contributed kernel that more closely matches what the raspberry Pi inc developers were doing.

mdevaev commented 3 years ago

Okay, if you want to go back to this issue, I won't mind.

faddat commented 3 years ago

OK, I'll leave it open-- when mainline smoothly supports Pi 4, it'll make good sense to add arm64 support, I reckon.

mdef commented 3 years ago

I'm also interested to have 64bit build or can you confirm if it possible to run 32bit packages what you provide on aarch64 arch linux?

mdevaev commented 3 years ago

I haven't tested this case, so I can't answer.

faddat commented 3 years ago

You can't run 32-bit packages on 64-bit arch, unless you want to play with qemu binfmt_misc.

But there are builds of everything in the default repositories though so you shouldn't really need to run 32-bit packages

mdef commented 3 years ago

I'm actually interested to have 64bit arch linux to run pikvm on it, and other sortware in docker. Will it work this way:

mdevaev commented 3 years ago

64bit is not compatible with omx encoder required for tc358743. perhaps something else is not working, I have not studied this issue in detail

faddat commented 3 years ago

I'm actually interested to have 64bit arch linux to run pikvm on it, and other sortware in docker. Will it work this way:

  • install ArchLinuxARM-rpi-aarch64-latest.tar.gz
  • install precompiled pikvm for aarch64 if the exists somewhere Thank you in advance

No, but it will work this way:

https://github.com/faddat/sos/blob/dcdfa6e029d683d38e379389f2fdad7f393bd0c0/Dockerfile#L29-L33

Those lines are in fact super important. Also, if you'd like, you can just try forking SOS which is more or less a derivative of pi-builder here in fact. The biggest difference between SOS and Pi builder is probably that I offer the user fewer options, and don't really do very much parameterization or use of variables. The build script is quite linear.

I'm also pretty confident that you can actually build SOS on your pi. Typically, I build SOS on an AWS graviton instance, the experience is just delightful.

@mdef I should also mention that I may have been wrong, you may actually be able to run 32 bit arm packages on your 64-bit arm CPU. Of course, your mileage may vary.

faddat commented 3 years ago

So the thing that I'd like to contribute I think is actually the use of Docker buildx.

I'm also going to look over that PR that adds multi-device support because I'm tinkering with the same stuff in SOS.

Zitt commented 2 years ago

Coming up on two years later and no progress :(

mdevaev commented 10 months ago

Hey guys, better late than never, right?

I have added basic aarch64 support. Not all stages work with them (there is no watchdog, for example), but in general it should work.

However, I caution against further use of pi-builder on x86_64. It will be better to use it on arm or aarch64, i.e. build Raspberry on Raspberry. The reason is that QEMU can fail unpredictably inside different installation scripts in the package manager, and this can lead to unpredictable behavior and generation of faulty OS images. The new QEMU works poorly, QEMU for AArch64 is even more unstable, and the only thing that works well now is old QEMU armv7 on x86_64. I have abandoned cross-assembly and use a cluster of several Raspberries for my purposes.

onedr0p commented 10 months ago

Wow that's awesome, can't wait download the image and flash it when it's all ready

mdevaev commented 9 months ago

I'm marking this as solved, since technically the ability to build ARM64 images has already been implemented. OS for PiKVM is a separate stage.

onedr0p commented 4 months ago

@mdevaev is there an issue for tracking building/publishing arm64 images for PiKVM? I would like to use the PiKVM for a few more network utilities/apps in my homelab but the armv7 arch is very limiting.

mdevaev commented 4 months ago

I'm not working on it right now, but I have some plans for this year.