search

canonical / microk8s

MicroK8s is a small, fast, single-package Kubernetes for datacenters and the edge.
https://microk8s.io
Apache License 2.0
8.4k stars 767 forks source link

Is it possible to install microk8s on ubuntu core? #2053

Closed lejsekt closed 8 months ago

lejsekt commented 3 years ago

Hi, is it possible to install microk8s on rabsperry pi 4 with ubuntu core 20 for arm64?

sudo snap install microk8s --classic error: snap "microk8s" requires classic confinement which is only available on classic systems

Thank you

balchua commented 3 years ago

I think Ubuntu Core supports only strict confinement.

lejsekt commented 3 years ago

And is it planned to support Ubuntu Core in the future?

ktsakalozos commented 3 years ago

Indeed Ubuntu Core needs strict confinement. There is a branch where we got most of MicroK8s strictly confined [1]. @joedborg has worked on this.

@lejsekt what would you need a strictly confined MicroK8s for? We would like your input to make a case to work on it. Thanks.

[1] https://github.com/ubuntu/microk8s/tree/feature/jdb%2Fstrict

lejsekt commented 3 years ago

@ktsakalozos Well, my use case is the following. I'd like to setup a fleet of IoT devices and manage them through microk8s. Ubuntu Core seems like a good fit as a base OS for these devices. So it'd be great to be able to turn these Ubuntu Core devices into microk8s nodes.

mikepurvis commented 3 years ago

I'm interested as well, though I don't have a great motivation either other than looking for the smallest, quickest setup for rolling out a bunch of these nodes.

dylancourt commented 3 years ago

+1 for this! I work at a smart building company and we're looking to replace yocto os while also having a pathway to kubernetes on embedded devices at Edge sites

balchua commented 3 years ago

@ktsakalozos im adding this to the wishlist. 😊

usrbinkat commented 3 years ago

microk8s on top of ubuntu core would simply solve the hardest challenges that I face with deployment and lifecycle of assembly plant robots and low power edge compute at 5G tower installations. I could enumerate the layers of features that enhance the end to end lifecycle if needed but both together would be the compelling story that I'm really looking for.

Please let me know if I can help get visibility to the importance of this work.

ncpe20012003 commented 3 years ago

Any progress? I have a need to run microk8s on UC 20.

robert-schmuck commented 3 years ago

Hi, I am also very interested in running mickrok8s on ubuntu core 20, is there plan when to implement this?

ncpe20012003 commented 3 years ago

Checking in, any word on MicroK8s in UC 20?

balchua commented 3 years ago

@ncpe20012003 i know there's been plenty of work being put on this. @ktsakalozos maybe able to share more on this.

ktsakalozos commented 3 years ago

We are actively working on getting a strictly confined snap. There is a branch where you could follow our progress https://github.com/ubuntu/microk8s/tree/feature/jdb%2Fstrict . The build for now go in the latest/edge/strict channel. You could give it a try with:

sudo snap install microk8s --channel=latest/edge/strict --devmode
Christoph02 commented 3 years ago

Also waiting on microk8s for ubuntu core. Tried on proxmox and ran in same issue. Would be great to have this very cool snap on ubuntu core, too. currently not working with --channel=latest/edge/strict even not in devmode.

Kind regards, Christoph

Ch4s3r commented 2 years ago

@ktsakalozos There was a typo in your command in mcirok8s. Works awesome otherwise on ubuntu core :)

sudo snap install microk8s --channel=latest/edge/strict --devmode
ktsakalozos commented 2 years ago

We would appreciate if you could run in non-devmode (without the --devmode) and provide us with some feedback on what may be broken. An inspection report should be enough (microk8s inspect) . Thank you.

Ch4s3r commented 2 years ago

There was an error on my side, I only tested it on a VM and expected it to also work on a PI. So it's currently not working on the Pi 4 directly (with or without devmode). Error message that I get on both from sudo microk8s inspect: microk8s is not running. Use microk8s inspect for a deeper inspection.

But anyway here are the two reports if they still help: devmode: inspection-report-20211004_213459.tar.gz non-devmode: inspection-report-20211004_205641.tar.gz

Both setup on a fresh ubuntu core 20 arm64 distro on a Pi 4.

ktsakalozos commented 2 years ago

@Ch4s3r in the logs I see this error:

Oct 04 21:34:40 ubuntu microk8s.daemon-kubelite[7068]: E1004 21:34:40.630574    7068 kubelet.go:1423] "Failed to start ContainerManager" err="system validation failed - Following Cgroup subsystem not mounted: [memory]"

Have you enabled cgroups as described in https://microk8s.io/docs/install-alternatives#heading--arm ?

Ch4s3r commented 2 years ago

I did but maybe in the wrong place. Because I could not find the firmware folder but there was another cmdline.txt somewhere in /writable/.../system-data/... Will have to recheck this in the evening.

Ch4s3r commented 2 years ago

Tried but could not append the kernel command line arguments on Ubuntu Core 20. There is no /boot/firmware folder to write them to and there is no possibility to create this folder as the system is read-only. Any suggestions how to do this?

Ch4s3r commented 2 years ago

Had to add cgroup_enable=memory cgroup_memory=1 to /ubuntu-seed/cmdline of the sd card.

Now it's successfully shown that cgroups are enabled:

> grep memory /proc/cgroups
memory  6       269     1

But still getting this:

> sudo microk8s status
microk8s is not running. Use microk8s inspect for a deeper inspection.

Here's another log (with devmode): inspection-report-20211006_201819.tar.gz

Ch4s3r commented 2 years ago

Somehow it started working after invoking sudo microk8s status multiple times or maybe it was also a reboot in-between.

Here the log of a working non-devmode microk8s on Ubuntu Core 20: inspection-report-20211006_210501.tar.gz 🥳

It also shows this on the status which is a bit worrying:

Fatal: can't open lock file /run/xtables.lock: Permission denied
/snap/microk8s/2556/inspect.sh: line 187: /bin/mount: Permission denied
/snap/microk8s/2556/inspect.sh: line 188: /bin/mount: Permission denied
WARNING:  The memory cgroup is not enabled.
The cluster may not be functioning properly. Please ensure cgroups are enabled

But again i verified that the memory cgroup is indeed enabled:

> grep memory /proc/cgroups
memory  6       280     1
Christoph02 commented 2 years ago

Thank you Konstantinos (https://github.com/ktsakalozos)

MicroK8s v1.25 released!

This is a real milestone: Strict confinement goes into general availability

It worked for me with Ubuntu Core on Raspberry 4.

snap install microk8s --channel=1.25-strict/stable
microk8s (1.25-strict/stable) v1.25.0 from Canonical✓ installed

The IoT experience with Ubuntu Core and MicroK8s can continue, now ...

Thanks a lot and best regards, Christoph

usrbinkat commented 2 years ago

This is exciting!

joedborg commented 2 years ago

So happy to see this go GA! Well done everyone.

edocalia commented 1 year ago

Thanks everyone, this is one major step beyond!

Duc commented 1 year ago

Hi, I successfully installed and run MicroK8s v1.26.0 revision 4393 on Ubuntu Core 22. I'm in a dead end now regarding NFS volume, It's seems that Ubuntu Core is not able to mount NFS share. Am I right ?

ktsakalozos commented 1 year ago

@Duc I asked over at https://forum.snapcraft.io/t/mount-nfs-share-on-ubuntu-core/33332 as this might be a topic for snaps on core in general.

stale[bot] commented 9 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.