Raspberry Pi images

[TheAifam5]

Is your feature request related to a problem? Please describe. I really wish to have the .img files for Raspberry Pi's, like RancherOS have, or some kind of installer for SDCards. K3S declares:

5. Minimal to no OS dependencies (just a sane kernel and cgroup mounts needed)...

would be much much lighter than installing on top of Debian or Ubuntu, also not bloated with systemd and other stuff which are not used by k3s.

Describe the solution you'd like I think, some stuff can be taken from RancherOS repository to achieve that feature.

[rharmonson]

I second the request.

I have three Raspberry PI 4 reproachfully staring at me from my desk for not having installed K3os

[solvaholic]

@TheAifam5 @rharmonson

I think, some stuff can be taken from RancherOS repository to achieve that feature.

I thought this also. The discussion in how do rancheros and k3os differ? https://github.com/rancher/k3os/issues/201 helped me see how the two OS are different.

Please be sure to :+1: your preferred Raspberry Pi in What ARM boards would the community like supported? https://github.com/rancher/k3os/issues/65.

I have three Raspberry PI 4 reproachfully staring at me from my desk for not having installed K3os

You may find the discussion in Arm ISO is not bootable for Raspberry Pi 3 https://github.com/rancher/k3os/issues/111 helpful, in particular https://github.com/rancher/k3os/issues/111#issuecomment-548001732.

[zimme]

I spent a bit of time looking into if there was a way to create an image that had u-boot which could boot k3os via grub2 via uefi from the arm64 iso (u-boot supports uefi).

I kinda got it booting, however I do believe there was something weird with the kernel or something as it wasn't able to boot properly.

If we could successfully do this, we could potentially create a script that would help you create an image where you could provide an url to a u-boot version that your board supports, defaulting to upstream u-boot, and it would also write the arm/arm64 iso to the image and it would almost be as easy as just downloading and image flashing it.

[zimme]

For this to work though, I guess the kernel used in k3os would need to be ubuntu's HWE kernel, which I know there's a PR for, or there would "need" to be a way to bring your own kernel for cases where you need a custom kernel to support your board.

[rharmonson]

HWE?

New term for me, so here is a link for others that may not be familiar with Ubuntu's "HardWare Enabled" kernel.

https://askubuntu.com/questions/248914/what-is-hardware-enablement-hwe

[rharmonson]

Stumbled onto this but not tested, yet. It may be something warranting exploration. Thoughts?

https://github.com/pftf/RPi4

[zimme]

Stumbled onto this but not tested, yet. It may be something warranting exploration. Thoughts?

https://github.com/pftf/RPi4

I've seen that too, I'd suggest we start with looking into u-boot as that also has an uefi interface too but also works on multiple boards and not just the rpi's.

If there's a problem getting u-boot working the way we need it on rpis that project might be a good alternative.

[rharmonson]

I agree with your suggestion.

I found a u-boot and raspberry specific article which I will spend time reviewing this weekend. This is not my area of expertise, but will take a stab at it in hopes I can contribute.

[zimme]

So far I've gotten u-boot up and running on some of my rpis, I'm able to load k3os iso via usb stick. I'm now looking into trying to get the same iso to load from a second partition on the sdcard.

[jdbohrman]

So far I've gotten u-boot up and running on some of my rpis, I'm able to load k3os iso via usb stick. I'm now looking into trying to get the same iso to load from a second partition on the sdcard.

Any luck with this?

[sgielen]

I created this repository to generate k3os images for the Raspberry Pi and another arm64 device: https://github.com/sgielen/picl-k3os-image-generator

[rharmonson]

@sgielen, thank you!

I reviewed your repository and it looks very promising. I do have a question regarding build-image.sh. Reviewing your code, I assume I would just update the line 78 with the URL from https://github.com/raspberrypi/firmware/releases when future releases occur?

78: if [ "$IMAGE_TYPE" = "raspberrypi" ]; then dl_dep raspberrypi-firmware.tar.gz https://github.com/raspberrypi/firmware/archive/1.20190925.tar.gz

As soon as I complete a proof of concept with RPi4s and Ceph, I will explore your work more completely.

Again, thank you!

[sgielen]

@rharmonson Yes, although currently you'd also need to remove the existing file to get it to download the new one. I'm considering fixing that by letting it check against the expected SHA sum or something like that.

Feel free to try it and submit a pull request if you find it works for you :-)

[rharmonson]

Thank you and I may very well give it a go.

[vnelis]

@sgielen Hi there, I just bought 4 RPi 4 to make my own cluster as well and I'd like to install k3os on them. The objective is to run a web application developed using the MEAN stack. Do you think that would work or should I use a light raspian as anyone does?

[vnelis]

I'm just afraid to waste many hours to eventually realize that what I'm trying is way to "hacky" and then go back to a more standard approach

[solvaholic]

@vnelis I feel ya. My Docker needs are simple, and I am loving swarm on RancherOS.

[sgielen]

@sgielen Hi there, I just bought 4 RPi 4 to make my own cluster as well and I'd like to install k3os on them. The objective is to run a web application developed using the MEAN stack. Do you think that would work or should I use a light raspian as anyone does?

Hi @vnelis! I would think that if the application isn't too heavy, and you are willing to invest some time in getting it running on Kubernetes on arm64, it will probably work well. Similar to you, I have three Pi-like devices in a cluster. Keep in mind that it's possible the latency is higher than with modern x86 systems.

In order to get the most out of a cluster setup like this (if only in terms of fun), I'd suggest looking into highly available storage like OpenEBS (which, with some work, seems to work well on arm64) and use a system like MetalLB to give your Kubernetes Service objects IPs in your network that don't correspond to a single node. This way, you eliminate SPOFs within your cluster, allowing you to perform maintenance on single nodes at a time without bringing down your application.

[nazarewk]

@sgielen do you have any instructions on setting up OpenEBS on Raspberry Pi 4 (i've 3x 4GB versions)? I've tried settings up cStor backend, but every time i try to use PVs i get defunct zrepl processes in cstor pools pods requiring reboot -fn on the affected nodes.

[PaulCharlton]

generally, arm32 build of all components is desirable (kernel, initrd, etc)

[reesericci]

I still can't believe that 2yrs later that there are still no official images for RPI.

[TheAifam5]

The rancher-os has not been updated for ages and looks like they remake everything from scratch. Maybe afterwards would be much easier to setup k3s - will see :) I'm still staying positive about k3os on raspi.

[reesericci]

They deprecated RancherOS a while ago. (probably in favor of k3os)

[TheAifam5]

They deprecated RancherOS a while ago. (probably in favor of k3os)

oh but what about the v2-test branch: https://github.com/rancher/os/tree/v2-test

[reesericci]

spicy