kairos-io / kairos

:penguin: The immutable Linux meta-distribution for edge Kubernetes.
https://kairos.io
Apache License 2.0
1.09k stars 97 forks source link

feat: Add k3s option for nvidia orin images #2900

Open laurivosandi opened 6 days ago

laurivosandi commented 6 days ago

Is your feature request related to a problem? Please describe.

No k3s images are available for nvidia-jetson-agx-orin

Describe the solution you'd like

I'd like to be able to use k3s on Orin boards

Describe alternatives you've considered

Tried to locally build with:

earthly -P +all-arm \
  --VARIANT=standard \
  --MODEL=nvidia-jetson-agx-orin \
  --FLAVOR=ubuntu \
  --FLAVOR_RELEASE=20.04 \
  --FAMILY=ubuntu \
  --BASE_IMAGE=quay.io/kairos/cache:nvidia-base \
  --IMG_COMPRESSION=false \
  --K3S_VERSION=1.31.1

It succeeded but it seems there is no way to locally use the built image with osbuilder-tools or luet? Screenshot from 2024-09-27 10-52-10

Additional context

mudler commented 2 days ago

@laurivosandi for Nvidia AGX we don't publish yet standard images with k3s.

For the moment it would be enough to use the images currently published (without k3s) and overlay on top the k3s/provider packages.

For instance, from our Dockerfiles, this is what standard images are including:

FROM base-kairos AS kairos-standard
ARG SOFTWARE_VERSION
LABEL io.kairos.k3s_version="${SOFTWARE_VERSION}"
RUN luet install -y system/provider-kairos
RUN luet install -y "k8s/k3s-$(which-init.sh)@${SOFTWARE_LUET_VERSION:-$SOFTWARE_VERSION}" utils/edgevpn utils/k9s utils/nerdctl container/kubectl utils/kube-vip

You can install k3s and provider-kairos with luet, or just overlay the container images with COPY / / --from=container-image