search

burmilla / os

Tiny Linux distro that runs the entire OS as Docker containers
https://burmillaos.org
Apache License 2.0
210 stars 13 forks source link

v2 branch updates? #134

Closed Raboo closed 1 year ago

Raboo commented 2 years ago

Hi

I've been running BurmillaOS v2.0.0-beta4 for quite some time now. However it seems that this project is focused on the v1 more. I would love to see docker engine and kernel updates to v2. Why aren't v2 moved into stable and getting more attention?

olljanat commented 2 years ago

Situation is that I tried to propose long term target on #88 but we wasn't able to agree that so idea was abandoned.

Current proposal about how to continue with v2.x versions can be found from #119 It is waiting for feedback and that Rancher gets their v2 platform to more stable state.

Third and most probably biggest issue is that this project is lacking of maintainers/contributors. We have #2 open about that discussion but I have not heard most of those guys for a while. So unless that changes I will keep updating those v1.9.x versions as long we run those in production and then this project will die (will give you guys heads up when I know schedule for that).

tredger commented 2 years ago

In lieu of decisions being made about the future direction of 2.x, what is your appetite for some pull requests to the various repos (os-kernel, os-base, os-initrd-base, os) to update the 5.10.x kernel and buildroot for the relevant branch/master. I'd also like to add wireguard support into kernel-extras.

olljanat commented 2 years ago

I'm happy to accept pull requests to those and release new beta version based on them.

If we stay on 5.10.x kernels then next version can be v2.0.0-beta5 and if we which to 5.15.x (latest LTS) then most probably we should call it for v2.1.0-beta1

When it comes to os-base/os-initrd-base notice that kernel-4.14.x branches are currently ahead master so there might be useful information what is needed with later buildroot versions (at least there was this issue with dhcpd https://github.com/burmilla/os-base/commit/5dd93a3a9c6ee38b3f8f1d760ecebbf5de02f368 )

Including Wireguard to kernel-extras can be done and depending size of those modules we can consider even include them to standard package.

tredger commented 2 years ago

Awesome. PRs incoming.

I've cherry-picked what I think are the relevant commits for both os-base/os-initrd-base kernel-4.14.x branches. I assumed that the changes would be the same as the dhcpd version isn't dependent on the kernel support. I'll start with 5.10.x as I've actually already built that and got it running. But I'll take a look at 5.15.x afterwards.

I'll start with os-kernel, os-base, and os-initrd-base so I can use the new released versions to test os once the github actions have been run.

tredger commented 2 years ago

56455886 Jul 20 22:57 linux-5.10.129-burmilla-x86.tar.gz with wireguard included in kernel 56448599 Jul 17 16:03 linux-5.10.129-burmilla-x86.tar.gz with wireguard not included (compiled into kernel-extras)

So including wireguard assumedly adds ~7kb. I assume this increase is OK? If so I will update the os-kernel PR to build it in.

Are you OK with the other two PRs os-initrd-base and os-base?

olljanat commented 2 years ago

So including wireguard assumedly adds ~7kb. I assume this increase is OK? If so I will update the os-kernel PR to build it in.

Yes, that is fine to be included to kernel.

Are you OK with the other two PRs os-initrd-base and os-base?

Oh, I had forgot enable notifications to those repos. Merged them now and triggered build.

olljanat commented 2 years ago

@tredger btw. I added you as read only member to burmilla organization. With that one you most should be able to see also draft versions of releases.

tredger commented 2 years ago

@tredger btw. I added you as read only member to burmilla organization. With that one you most should be able to see also draft versions of releases.

@olljanat Thanks. I've accepted the invite. Though I can't see any draft releases (unless I'm missing something, or the release hasn't been drafted yet?).

olljanat commented 2 years ago

FYI. I just marked v2.0.0-beta5 to public as I cannot find any very critical issues from it can after all we still call it for beta.

Raboo commented 2 years ago

@tredger thanks for the great work. I guess we can close this ticket then and hopefully @tredger keeps the v2 branch maintained. Perhaps in the near future he will become a full member of the burmilla github organisation.

(Unfortunately I already moved away from BurmillaOS into Flatcar Linux, but I still believe that this OS still has a future if it gets a company that is willing to put some time into it or enough hobbyists or freelancers).

olljanat commented 2 years ago

Re-open to continue discussion about topic.

@Raboo may I ask that why you ended up to Flatcar Linux? I'm not familiar with that one.

Raboo commented 2 years ago

I'm running rancher rke1 (kubernetes in docker) and I am running rook-ceph(ceph in container). This makes me need an updated kernel. At the time of the decision BurmillaOS 2.0 (1.x has to old kernel for ceph) hadn't been updated in a very long while. No new docker version, no kernel upgrade. So I had a situation where I was getting segfaults and wanted to update. And unfortunately I don't really have the time to become an maintainer for BurmillaOS.

In reality there aren't so many Linux OS's that are easy to network boot with cloud-init or similar and almost immutable. So I decided to go for Flatcar Linux. It's a fork of CoreOS, it was maintained by a company that is called Kinovolk and specializes in Kubernetes. Kinovolk got bought by Microsoft, AFAIK they plan to keep maintaining Flatcar.

So with Flatcar the learning curve is higher since it uses something called ignition instead of cloud-init. It's a bit more complicate to setup. But still it fits my criteria of having a config that is applied on boot. The difference is that it's designed to be applied on first boot only. Which for some it's a plus, for some a downside. However it did work for my use case. And if I do want a config that is applied every boot, I can simply set the nodes to start from a clean slate every boot. With Flatcar I get a OS that is backed by a big company. Where all the components are updated (ex docker, containerd, cri-o, cgroups, kernel).

It has longer history and a bigger community, I did get stuck during the testing phase and I did get help from the community.

Flatcar is more of a complete OS while Burmilla more lightweight. For me being lightweight means simple which is a plus. But I need to be able to rely on getting updates and some form of community activity.

pwFoo commented 1 year ago

Hi @olljanat I would like to test v2.0.0-beta6. Would it possible to upgrade and optional downgrade if I get trouble with my used services?

olljanat commented 1 year ago

Upgrade to it should works on same way like upgrade RancherOS https://burmillaos.org/docs/installation/upgrading/#upgrade-rancheros-to-burmillaos

Just use https://raw.githubusercontent.com/burmilla/releases/master/releases.yml

I have not tested downgrade but at least on theory it should works same way.

pwFoo commented 1 year ago

Hi @olljanat, I have currently still waited with the update, also due to little time for debugging in case of problems. In the meantime beta7 has been released. How close are we to a stable release here?

olljanat commented 1 year ago

In the meantime beta7 has been released.

There have been multiple iterations of beta7 but those are not public (except the code of course). When it is released it appears to https://github.com/burmilla/os/releases

How close are we to a stable release here?

It is more about that which changes we want to include before marking it release candidate because after that only bug fixes can happen and all bigger changes must go to 2.1.x or 3.0.x versions.

Most critical open topics are:

pwFoo commented 1 year ago

Updated to beta6 for testing... First looks it works fine.

Upgrade fixed a nodejs app update problem for me, but still problem with newer nodejs apps running in user docker container.

olljanat commented 1 year ago

Most critical open topics are:

* Support AppArmor instead of Selinux (which have been experimental feature anyway) and with that one I'm currently working on.

* Using balena-engine instead of system-docker [Update or replace system-docker #28 (comment)](https://github.com/burmilla/os/issues/28#issuecomment-1423331985) (or decide not to do it but I have not given up yet)

On latest draft AppArmor looks to be working and be safe to be enabled by default 🚀

One new item to list however, it would be nice to enable /sys/kernel/btf/vmlinux to kernel because it adds support for tools like Tracee. Sad thing is that it makes kernel package a lot of bigger:

Without BTF: linux-5.10.176-burmilla-x86.tar.gz  138 MB
With BTF:  linux-5.10.177-burmilla-x86.tar.gz   679 MB

Tracee looks to support also importing BTF files from separate file but I'm not sure if that works with other tools.

olljanat commented 1 year ago

v2.0.0-beta7 is now available and I would appreciate if someone have time test it properly and provide feedback?

olljanat commented 1 year ago

Closing because v2.0.0-rc1 was released and I created separate issue tracker to #161 finalizing it before GA version release.