aenix-io / kubefarm

Automated Kubernetes deployment and the PXE-bootable servers farm
Apache License 2.0
333 stars 27 forks source link

Multiarch support? #3

Open sfxworks opened 3 years ago

sfxworks commented 3 years ago

I am testing out a cluster that uses both arm64 and x64. How would one go about configuring this to serve the proper images depending on what is detected?

I was trying to start as looking how it is done with dnsmasq at the same time as building an arm variant of the image. Though I run into errors:

ryzen $ docker buildx build --platform linux/arm64 -t quantomworks/k8s-image-test:latest --push .
[+] Building 258.1s (11/29)
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                   0.1s 
 => => transferring dockerfile: 32B                                                                                                                                                                                                                    0.0s 
 => [internal] load .dockerignore                                                                                                                                                                                                                      0.1s 
 => => transferring context: 2B                                                                                                                                                                                                                        0.0s 
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                                                                                                                                                                        1.5s 
 => [internal] load build context                                                                                                                                                                                                                      0.1s 
 => => transferring context: 331B                                                                                                                                                                                                                      0.0s 
 => [ltsp 1/4] FROM docker.io/library/ubuntu:20.04@sha256:b4f9e18267eb98998f6130342baacaeb9553f136142d40959a1b46d6401f0f2b                                                                                                                             2.8s 
 => => resolve docker.io/library/ubuntu:20.04@sha256:b4f9e18267eb98998f6130342baacaeb9553f136142d40959a1b46d6401f0f2b                                                                                                                                  0.1s 
 => => sha256:a361e87bde5e27ceda037ab2116d88d9570b6d53c34102a2a214ef2944270138 188B / 188B                                                                                                                                                             0.1s 
 => => sha256:e5be16fdc30614524222cdec466297e7ed49c7695e868c5dd2700a1778d88b23 853B / 853B                                                                                                                                                             0.1s 
 => => sha256:32d7611b468cbc07986302f1a8e74d92e30a1f11cdfa8bc2900aedda2758d050 27.18MB / 27.18MB                                                                                                                                                       2.0s 
 => => extracting sha256:32d7611b468cbc07986302f1a8e74d92e30a1f11cdfa8bc2900aedda2758d050                                                                                                                                                              0.6s 
 => => extracting sha256:e5be16fdc30614524222cdec466297e7ed49c7695e868c5dd2700a1778d88b23                                                                                                                                                              0.1s 
 => => extracting sha256:a361e87bde5e27ceda037ab2116d88d9570b6d53c34102a2a214ef2944270138                                                                                                                                                              0.1s 
 => [ltsp 2/4] RUN apt-get -y update  && apt-get -y upgrade  && apt-get -y install       curl  && apt-get clean                                                                                                                                       97.3s 
 => [ltsp 3/4] RUN printf '%s\n'       'deb http://ppa.launchpad.net/ltsp/ppa/ubuntu focal main'       'deb http://ppa.launchpad.net/ltsp/proposed/ubuntu focal main'       > /etc/apt/sources.list.d/ltsp.list  && curl -L https://ltsp.org/misc/lts  1.7s 
 => [ltsp 4/4] RUN apt-get -y update  && apt-get -y install       ltsp-cloud  && apt-get clean                                                                                                                                                       131.1s 
 => CANCELED [rootfs-pre 1/7] RUN echo 'APT::Install-Recommends "0";\nAPT::Install-Suggests "0";'       >> /etc/apt/apt.conf.d/01norecommend  && mkdir -p /var/lib/resolvconf  && touch /var/lib/resolvconf/linkified  && apt-get update  && apt-get  23.1s 
 => CANCELED [builder 1/2] RUN apt-get -y update  && apt-get -y install       git       build-essential  && apt-get clean  && rm -rf /var/lib/apt/lists/* ARG DNSMAQ_VERSION=2.81-12-g619000a                                                         22.8s 
 => ERROR [stage-5 1/5] RUN apt-get -y update  && apt-get -y install       grub-pc-bin       grub-efi-amd64-bin       inotify-tools       nginx  && apt-get clean  && rm -rf /var/lib/apt/lists/*                                                     22.5s 
------
 > [stage-5 1/5] RUN apt-get -y update  && apt-get -y install       grub-pc-bin       grub-efi-amd64-bin       inotify-tools       nginx  && apt-get clean  && rm -rf /var/lib/apt/lists/*:
#8 1.518 Hit:1 http://ppa.launchpad.net/ltsp/ppa/ubuntu focal InRelease
#8 1.519 Hit:2 http://ports.ubuntu.com/ubuntu-ports focal InRelease
#8 1.548 Hit:3 http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease
#8 1.624 Hit:4 http://ppa.launchpad.net/ltsp/proposed/ubuntu focal InRelease
#8 1.652 Hit:5 http://ports.ubuntu.com/ubuntu-ports focal-backports InRelease
#8 1.741 Hit:6 http://ports.ubuntu.com/ubuntu-ports focal-security InRelease
#8 10.89 Reading package lists...
#8 16.15 Reading package lists...
#8 21.08 Building dependency tree...
#8 22.00 Reading state information...
#8 22.06 Package grub-efi-amd64-bin is not available, but is referred to by another package.
#8 22.06 This may mean that the package is missing, has been obsoleted, or
#8 22.06 is only available from another source
#8 22.06
#8 22.06 Package grub-pc-bin is not available, but is referred to by another package.
#8 22.06 This may mean that the package is missing, has been obsoleted, or
#8 22.06 is only available from another source
#8 22.06 
#8 22.31 E: Package 'grub-pc-bin' has no installation candidate
#8 22.31 E: Package 'grub-efi-amd64-bin' has no installation candidate
------
Dockerfile:197
--------------------
 196 |
 197 | >>> RUN apt-get -y update \
 198 | >>>  && apt-get -y install \
 199 | >>>       grub-pc-bin \
 200 | >>>       grub-efi-amd64-bin \
 201 | >>>       inotify-tools \
 202 | >>>       nginx \
 203 | >>>  && apt-get clean \
 204 | >>>  && rm -rf /var/lib/apt/lists/*
 205 |
--------------------
error: failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c apt-get -y update  && apt-get -y install       grub-pc-bin       grub-efi-amd64-bin       inotify-tools       nginx  && apt-get clean  && rm -rf /var/lib/apt/lists/*]: exit code: 100

I could probably knock this out by referencing other packages, but I am still stuck on how to prepare the configuration files for dnsmasq after studying this documentation and rpi's documentation.

kvaps commented 3 years ago

Hi, I'm not sure if grub has support for arm. But you can refer official LTSP documentation and try using iPXE instead.

FYI, iPXE is official booting method for LTSP, we use grub as it works better in certain firmwares and efi in our case. You would probably need to modify original Dockerfile and the manifests for LTSP server, but It would be amazing to support both methods.

So in case of success, feel free free to send PR or at least success story to this project :wink:

mrybas commented 3 years ago

for raspberries it should work like this

dhcp-mac=set:rpi,b8:27:eb:*:*:*
dhcp-mac=set:rpi,dc:a6:32:*:*:*
pxe-service=tag:rpi,X86PC,"Raspberry Pi Boot   ",unused