search

weaveworks / ignite

Ignite a Firecracker microVM
https://ignite.readthedocs.org
Apache License 2.0
3.49k stars 224 forks source link

Fix #872, Adding Almalinux and Rocky Linux support #879

Closed srbala closed 2 years ago

srbala commented 2 years ago

Replacing PR #876, to keep history clean.

Adding Almalinux and Rocky Linux support, Resolves Fix #872

srbala commented 2 years ago

@darkowlzz created new pew PR for clean history

darkowlzz commented 2 years ago

Thanks for updating. This seems to work for me now. Tested on amd64 ignite. I'll wait a while and see if I can test the arm64 images soon. Thanks again for contributing this.

I am having trouble testing locally with make. I was able to build with digest in Dockerfile locally. https://github.com/weaveworks/ignite/pull/879#discussion_r737978184

Can you share more details about your trouble with the build? Error logs of the build would be helpful to understand better. We use https://github.com/multiarch/qemu-user-static in the multi-arch build, if that helps. Also, you can join the weave community slack https://weave-community.slack.com/messages/ignite/ if you like discussing over slack.

srbala commented 2 years ago

Can you share more details about your trouble with the build? Error logs of the build would be helpful to understand better.

MacOS inline edit using sed -i is not finding the file. It works when using sed -i '' like this one. There are some stackoverflow discussions on this sed behavior. May be separate Issue and PR to track this one.? https://github.com/weaveworks/ignite/blob/0519ec546f776bf1f95f15ffbcfd0ada94fbcc9f/images/Makefile#L86-L88

srbala commented 2 years ago

Here in RockyLinux build log using make

$ cd ignitew/images 
$ make build-rockylinux
make build WHAT=rockylinux      RELEASE=8                                        IS_LATEST=true  IS_MANIFEST_LIST=1  GOARCH=amd64  DIGEST=f0d7460b97156f6c8ea2ae73152bc11fe410d272387d60ddff36dfcea22ef689
make[1]: Entering directory '/Volumes/MacWork/orgs/srbala/alma/ignitew/images'
# Register /usr/bin/qemu-ARCH-static as the handler for non-x86 binaries in the kernel
docker run --rm --privileged multiarch/qemu-user-static:register --reset
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k
cp -a rockylinux/. /var/folders/t_/q0sbhnk10vj7gpgfhs66g42w0000gn/T/ignite-image-build.O5e83JTMqn/
sed "s|QEMUARCH||g;s|DOCKERARCH|amd64|g" rockylinux/Dockerfile > /var/folders/t_/q0sbhnk10vj7gpgfhs66g42w0000gn/T/ignite-image-build.O5e83JTMqn/Dockerfile
sed -i '' "/COPY qemu/d" /var/folders/t_/q0sbhnk10vj7gpgfhs66g42w0000gn/T/ignite-image-build.O5e83JTMqn/Dockerfile
docker build \
        --build-arg RELEASE \
        --build-arg BINARY_REF \
        --build-arg GOARCH \
        --build-arg DIGEST \
        -t weaveworks/ignite-rockylinux:8-v0.9.0-105-ga706dab3-amd64 \
        /var/folders/t_/q0sbhnk10vj7gpgfhs66g42w0000gn/T/ignite-image-build.O5e83JTMqn
[+] Building 0.1s (11/11) FINISHED                                                                                                        
 => [internal] load build definition from Dockerfile                                                                                 0.0s
 => => transferring dockerfile: 1.12kB                                                                                               0.0s
 => [internal] load .dockerignore                                                                                                    0.0s
 => => transferring context: 2B                                                                                                      0.0s
 => [internal] load metadata for docker.io/rockylinux/rockylinux@sha256:f0d7460b97156f6c8ea2ae73152bc11fe410d272387d60ddff36dfcea22  0.0s
 => [internal] load build context                                                                                                    0.0s
 => => transferring context: 32B                                                                                                     0.0s
 => [1/6] FROM docker.io/rockylinux/rockylinux@sha256:f0d7460b97156f6c8ea2ae73152bc11fe410d272387d60ddff36dfcea22ef689               0.0s
 => CACHED [2/6] COPY resolv.conf /etc/                                                                                              0.0s
 => CACHED [3/6] RUN dnf -y install --setopt=install_weak_deps=False --setopt=tsflags=nodocs         iproute         iputils         0.0s
 => CACHED [4/6] RUN echo "" > /etc/machine-id && echo "" > /var/lib/dbus/machine-id                                                 0.0s
 => CACHED [5/6] RUN sed -i -e 's/^AcceptEnv LANG LC_\*$/#AcceptEnv LANG LC_*/' /etc/ssh/sshd_config                                 0.0s
 => CACHED [6/6] RUN echo "root:root" | chpasswd                                                                                     0.0s
 => exporting to image                                                                                                               0.0s
 => => exporting layers                                                                                                              0.0s
 => => writing image sha256:6eff7c3314c1d7974275ebc9f0f43bd6eab5139bb7136d2da5eec57749d8f5e9                                         0.0s
 => => naming to docker.io/weaveworks/ignite-rockylinux:8-v0.9.0-105-ga706dab3-amd64                                                 0.0s
find /var/folders/t_/q0sbhnk10vj7gpgfhs66g42w0000gn/T/ignite-image-build.O5e83JTMqn/ -mindepth 1 -delete
docker tag weaveworks/ignite-rockylinux:8-v0.9.0-105-ga706dab3-amd64 weaveworks/ignite-rockylinux:8-amd64
docker tag weaveworks/ignite-rockylinux:8-v0.9.0-105-ga706dab3-amd64 weaveworks/ignite-rockylinux:latest-amd64
make[1]: Leaving directory '/Volumes/MacWork/orgs/srbala/alma/ignitew/images'
make build WHAT=rockylinux      RELEASE=8                                        IS_LATEST=true  IS_MANIFEST_LIST=1  GOARCH=arm64  DIGEST=e98dd481db7973d701f8695df7be1b2393261cf273aa68a89d873b7b134588df
make[1]: Entering directory '/Volumes/MacWork/orgs/srbala/alma/ignitew/images'
# Register /usr/bin/qemu-ARCH-static as the handler for non-x86 binaries in the kernel
docker run --rm --privileged multiarch/qemu-user-static:register --reset
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
Setting /usr/bin/qemu-ppc64le-static as binfmt interpreter for ppc64le
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k
Setting /usr/bin/qemu-mips-static as binfmt interpreter for mips
Setting /usr/bin/qemu-mipsel-static as binfmt interpreter for mipsel
Setting /usr/bin/qemu-mipsn32-static as binfmt interpreter for mipsn32
Setting /usr/bin/qemu-mipsn32el-static as binfmt interpreter for mipsn32el
Setting /usr/bin/qemu-mips64-static as binfmt interpreter for mips64
Setting /usr/bin/qemu-mips64el-static as binfmt interpreter for mips64el
Setting /usr/bin/qemu-sh4-static as binfmt interpreter for sh4
Setting /usr/bin/qemu-sh4eb-static as binfmt interpreter for sh4eb
Setting /usr/bin/qemu-s390x-static as binfmt interpreter for s390x
Setting /usr/bin/qemu-aarch64-static as binfmt interpreter for aarch64
Setting /usr/bin/qemu-aarch64_be-static as binfmt interpreter for aarch64_be
Setting /usr/bin/qemu-hppa-static as binfmt interpreter for hppa
Setting /usr/bin/qemu-riscv32-static as binfmt interpreter for riscv32
Setting /usr/bin/qemu-riscv64-static as binfmt interpreter for riscv64
Setting /usr/bin/qemu-xtensa-static as binfmt interpreter for xtensa
Setting /usr/bin/qemu-xtensaeb-static as binfmt interpreter for xtensaeb
Setting /usr/bin/qemu-microblaze-static as binfmt interpreter for microblaze
Setting /usr/bin/qemu-microblazeel-static as binfmt interpreter for microblazeel
Setting /usr/bin/qemu-or1k-static as binfmt interpreter for or1k
cp -a rockylinux/. /var/folders/t_/q0sbhnk10vj7gpgfhs66g42w0000gn/T/ignite-image-build.Vbm9mpwvyo/
sed "s|QEMUARCH|aarch64|g;s|DOCKERARCH|arm64v8|g" rockylinux/Dockerfile > /var/folders/t_/q0sbhnk10vj7gpgfhs66g42w0000gn/T/ignite-image-build.Vbm9mpwvyo/Dockerfile
make -C .. qemu
make[2]: Entering directory '/Volumes/MacWork/orgs/srbala/alma/ignitew'
make[2]: Nothing to be done for 'qemu'.
make[2]: Leaving directory '/Volumes/MacWork/orgs/srbala/alma/ignitew'
cp ../bin/arm64/qemu-aarch64-static /var/folders/t_/q0sbhnk10vj7gpgfhs66g42w0000gn/T/ignite-image-build.Vbm9mpwvyo
docker build \
        --build-arg RELEASE \
        --build-arg BINARY_REF \
        --build-arg GOARCH \
        --build-arg DIGEST \
        -t weaveworks/ignite-rockylinux:8-v0.9.0-105-ga706dab3-arm64 \
        /var/folders/t_/q0sbhnk10vj7gpgfhs66g42w0000gn/T/ignite-image-build.Vbm9mpwvyo
[+] Building 51.2s (13/13) FINISHED                                                                                                       
 => [internal] load build definition from Dockerfile                                                                                 0.0s
 => => transferring dockerfile: 1.16kB                                                                                               0.0s
 => [internal] load .dockerignore                                                                                                    0.0s
 => => transferring context: 2B                                                                                                      0.0s
 => [internal] load metadata for docker.io/rockylinux/rockylinux@sha256:e98dd481db7973d701f8695df7be1b2393261cf273aa68a89d873b7b134  1.0s
 => [auth] rockylinux/rockylinux:pull token for registry-1.docker.io                                                                 0.0s
 => [internal] load build context                                                                                                    0.2s
 => => transferring context: 4.49MB                                                                                                  0.1s
 => CACHED [1/7] FROM docker.io/rockylinux/rockylinux@sha256:e98dd481db7973d701f8695df7be1b2393261cf273aa68a89d873b7b134588df        0.0s
 => [2/7] COPY qemu-aarch64-static /usr/bin/                                                                                         0.1s
 => [3/7] COPY resolv.conf /etc/                                                                                                     0.0s
 => [4/7] RUN dnf -y install --setopt=install_weak_deps=False --setopt=tsflags=nodocs         iproute         iputils         open  48.4s
 => [5/7] RUN echo "" > /etc/machine-id && echo "" > /var/lib/dbus/machine-id                                                        0.4s
 => [6/7] RUN sed -i -e 's/^AcceptEnv LANG LC_\*$/#AcceptEnv LANG LC_*/' /etc/ssh/sshd_config                                        0.4s
 => [7/7] RUN echo "root:root" | chpasswd                                                                                            0.4s
 => exporting to image                                                                                                               0.2s
 => => exporting layers                                                                                                              0.2s
 => => writing image sha256:e892161e2dc8fa30c5a2ff3d57d74905a990eee48ce4b8bc843c4db2e1232185                                         0.0s
 => => naming to docker.io/weaveworks/ignite-rockylinux:8-v0.9.0-105-ga706dab3-arm64                                                 0.0s
find /var/folders/t_/q0sbhnk10vj7gpgfhs66g42w0000gn/T/ignite-image-build.Vbm9mpwvyo/ -mindepth 1 -delete
docker tag weaveworks/ignite-rockylinux:8-v0.9.0-105-ga706dab3-arm64 weaveworks/ignite-rockylinux:8-arm64
docker tag weaveworks/ignite-rockylinux:8-v0.9.0-105-ga706dab3-arm64 weaveworks/ignite-rockylinux:latest-arm64
make[1]: Leaving directory '/Volumes/MacWork/orgs/srbala/alma/ignitew/images'
darkowlzz commented 2 years ago

The build logs looks good with the sed fix.

MacOS inline edit using sed -i is not finding the file. It works when using sed -i '' like this one. There are some stackoverflow discussions on this sed behavior. May be separate Issue and PR to track this one.?

Yes, a separate issue and PR to address it would be better. Although ignite is a linux only project, but if we can enable building images on MacOS, that'd be nice to have.

I got a chance to test the arm64 images on a raspberry-pi with ubuntu 20.04, with KVM support. Here are some results:

Created a VM using the almalinux image:

[root@4465d93caddbe680 ~]# cat /etc/os-release
NAME="AlmaLinux"
VERSION="8.4 (Electric Cheetah)"
ID="almalinux"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.4"
PLATFORM_ID="platform:el8"
PRETTY_NAME="AlmaLinux 8.4 (Electric Cheetah)"
ANSI_COLOR="0;34"
CPE_NAME="cpe:/o:almalinux:almalinux:8.4:GA"
HOME_URL="https://almalinux.org/"
DOCUMENTATION_URL="https://wiki.almalinux.org/"
BUG_REPORT_URL="https://bugs.almalinux.org/"

ALMALINUX_MANTISBT_PROJECT="AlmaLinux-8"
ALMALINUX_MANTISBT_PROJECT_VERSION="8.4"

[root@4465d93caddbe680 ~]# uname -a
Linux 4465d93caddbe680 5.10.51 #1 SMP Mon Jul 19 19:27:37 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

Created a VM with rockylinux image:

[root@f8c437607a9da8e0 ~]# cat /etc/os-release
NAME="Rocky Linux"
VERSION="8.4 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel fedora"
VERSION_ID="8.4"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.4 (Green Obsidian)"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:rocky:rocky:8.4:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
ROCKY_SUPPORT_PRODUCT="Rocky Linux"
ROCKY_SUPPORT_PRODUCT_VERSION="8"
[root@f8c437607a9da8e0 ~]# uname -a
Linux f8c437607a9da8e0 5.10.51 #1 SMP Mon Jul 19 19:27:37 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

Everything seems to work well :)

darkowlzz commented 2 years ago

@srbala Thanks a lot for these images. The official ignite OS images are usually published with an ignite release. But we may publish it separately as well if there's a demand for it.

srbala commented 2 years ago

@darkowlzz Thanks for PR Merge. Since only two more months left to CentOS EOL, it would nice the alternates available soon. I am working on POC for bare metal install with ignite, official release will be helpful.

darkowlzz commented 2 years ago

Thanks for PR Merge. Since only two more months left to CentOS EOL, it would nice the alternates available soon. I am working on POC for bare metal install with ignite, official release will be helpful.

Okay, will try to publish these images soon.

darkowlzz commented 2 years ago

@srbala the official images are ready.

And the support for building on macOS was also added in https://github.com/weaveworks/ignite/pull/882 .

Thanks again for your contribution.

srbala commented 2 years ago

Thank you

stealthybox commented 2 years ago

Thanks to you as well @srbala !