search

lf-edge / eve

EVE is Edge Virtualization Engine
https://www.lfedge.org/projects/eve/
Apache License 2.0
479 stars 164 forks source link

Missing Docker Image in RP4 build? (Again? :) #4404

Open PhilienTaylor opened 1 month ago

PhilienTaylor commented 1 month ago

Describe the bug

Hi! I can't wait to use EVE. And Adam. For good. I'm building your Raspberry Pi 4 image for the first time, and I'm getting responses from the server similar to #1600 ... except for the fact it's 4 years later? And the image my system is trying to build with has 'dirty' in the name? :/

Recent release

Is it reproducible on a recent release? (See the list of releases)

No, I don't think so--I just did a straight git checkout yesterday ala the basic README instructions. B-)

Duplicate

If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment pointing to the existing issue instead.

How to reproduce

Include a list of steps showing how to reproduce the bug. Also include:

Expected behavior

I expected to have everything come out OK! :( I wanted the basic IMG file I could burn to my pi! I'm still new, so I had to do things like change the permissions, run as sudo, and add steps from preceding parts of the "README" that weren't included in

Error message

I tried the following commands before issuing the build command:

sudo docker system prune --all
sudo rm -rf dist/arm64
sudo docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
sudo docker run --privileged --rm tonistiigi/binfmt --install all
sudo docker pull lfedge/eve-dom0-ztools:latest-arm64

But just about every time..

itsmeusername@oops-hostname:~/my-project-dir/eve$ sudo make ZARCH=arm64 HV=xen live-raw
Makefile:121: "WARNING: We are assembling an arm64 image on amd64. Things may break."
# sample output 0.0.0-HEAD-a437e8e4-xen-amd64
mkdir -p /var/home/itsmeusername/my-project-dir/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.10.49/installer/
/var/home/itsmeusername/my-project-dir/eve/build-tools/bin/linuxkit pkg build --pull --platforms linux/arm64 pkg/u-boot # running linuxkit pkg build _without_ force ensures that we either pull it down or build it.
Building "lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8"
checking for docker.io/lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8 in local cache...
found docker.io/lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8 in local cache, skipping build
Build complete, not pushing, all done.
cd /var/home/itsmeusername/my-project-dir/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.10.49/installer/ && /var/home/itsmeusername/my-project-dir/eve/build-tools/bin/linuxkit cache export --arch arm64 --format filesystem --outfile - lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8 | tar xvf - boot
boot
boot/bcm2711-rpi-4-b.dtb
boot/config.txt
boot/fixup4.dat
boot/overlays
boot/overlays/bcm2711-spi-tpm-slb9670.dtbo
boot/overlays/raspberrypi-uno-220.dtbo
boot/start4.elf
boot/startup.nsh
boot/u-boot.bin
mkdir -p /var/home/itsmeusername/my-project-dir/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.10.49/installer/
/var/home/itsmeusername/my-project-dir/eve/build-tools/bin/linuxkit pkg build --pull --platforms linux/arm64 pkg/grub # running linuxkit pkg build _without_ force ensures that we either pull it down or build it.
Building "lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9"
checking for docker.io/lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9 in local cache...
found docker.io/lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9 in local cache, skipping build
Build complete, not pushing, all done.
cd /var/home/itsmeusername/my-project-dir/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.10.49/installer/ && /var/home/itsmeusername/my-project-dir/eve/build-tools/bin/linuxkit cache export --arch arm64 --format filesystem --outfile - lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9 | tar xvf - EFI
EFI
EFI/BOOT
EFI/BOOT/grub.cfg
EFI/BOOT/BOOTAA64.EFI
Done building /var/home/itsmeusername/my-project-dir/eve/build-tools/bin/linuxkit
Unable to find image 'mikefarah/yq:4.40.5' locally
4.40.5: Pulling from mikefarah/yq
661ff4d9561e: Pull complete 
8d593346e867: Pull complete 
964409861686: Pull complete 
ae1f0fe2115e: Pull complete 
df44078d133b: Pull complete 
Digest: sha256:32be61dc94d0acc44f513ba69d0fc05f1f92c2e760491f2a27e11fc13cde6327
Status: Downloaded newer image for mikefarah/yq:4.40.5
./tools/makerootfs.sh tar  -y images/out/rootfs-xen-generic.yml -t /var/home/itsmeusername/my-project-dir/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.10.49/rootfs.tar -d /var/home/itsmeusername/my-project-dir/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.10.49/installer -a arm64
Extract kernel image: docker.io/lfedge/eve-kernel:eve-kernel-arm64-v6.1.112-generic-235a293632c7-gcc
Image docker.io/lfedge/eve-kernel:eve-kernel-arm64-v6.1.112-generic-235a293632c7-gcc arch arm64 found in local cache, not pulling
Add init containers:
Process init image: docker.io/linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
Image docker.io/linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3 arch arm64 found in local cache, not pulling
Process init image: docker.io/linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
Image docker.io/linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f arch arm64 found in local cache, not pulling
Process init image: docker.io/linuxkit/containerd:95d5f0d2d8dc63bd87e96b7b39cf026cb86125c9
Image docker.io/linuxkit/containerd:95d5f0d2d8dc63bd87e96b7b39cf026cb86125c9 arch arm64 found in local cache, not pulling
Process init image: docker.io/linuxkit/getty:e74e6cad132403d1a6d6cd25b136a7c69c99f3f7
Image docker.io/linuxkit/getty:e74e6cad132403d1a6d6cd25b136a7c69c99f3f7 arch arm64 found in local cache, not pulling
Process init image: docker.io/linuxkit/memlogd:1ded209c4cc10aa8de2099f4156164b59df14e3c
Image docker.io/linuxkit/memlogd:1ded209c4cc10aa8de2099f4156164b59df14e3c arch arm64 found in local cache, not pulling
Process init image: docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64
Image docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64 arch arm64 incomplete or invalid in local cache, error no such reference: docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64, pulling
Error: failed to build init tarball from docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: could not pull image docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: error getting manifest for trusted image docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: GET https://index.docker.io/v2/lfedge/eve-dom0-ztools/manifests/400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: MANIFEST_UNKNOWN: manifest unknown; unknown tag=400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64
2024/10/26 06:49:58 error during command execution: failed to build init tarball from docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: could not pull image docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: error getting manifest for trusted image docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: GET https://index.docker.io/v2/lfedge/eve-dom0-ztools/manifests/400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: MANIFEST_UNKNOWN: manifest unknown; unknown tag=400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64
make: *** [Makefile:696: /var/home/itsmeusername/my-project-dir/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.10.49/rootfs.tar] Error 1
rm images/out/rootfs-xen-generic.yml.in

Additional context

I'm still new, so I had to do things like change the permissions, run as sudo, and add steps from preceding parts of the "README" that weren't included in the specific Raspberry Pi section B) :melting_face:

I am so happy to have found EVE! You all have no NO idea!!

PhilienTaylor commented 1 month ago

I'm running Fedora SilverBlue so give me a bit to make sure I didn't just hastily install Docker the wrong way.

PhilienTaylor commented 1 month ago

I can get Docker's Hello World running as a user, I'm not running any make as sudo (haha), and even if I need weird compensation for config.img permissions, I can get up to the same step:

myusername@myhostname:~/path-to-project/eve$ make ZARCH=arm64 HV=xen live-raw
Makefile:121: "WARNING: We are assembling an arm64 image on amd64. Things may break."
# sample output 0.0.0-HEAD-a437e8e4-xen-amd64
mkdir -p /var/home/myusername/path-to-project/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.13.50/installer/
/var/home/myusername/path-to-project/eve/build-tools/bin/linuxkit pkg build --pull --platforms linux/arm64 pkg/u-boot # running linuxkit pkg build _without_ force ensures that we either pull it down or build it.
Building "lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8"
checking for docker.io/lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8 in local cache...
docker.io/lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8 arm64 not found in local cache, trying to pull
INFO[0000] Image docker.io/lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8 arch arm64 incomplete or invalid in local cache, error no such reference: docker.io/lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8, pulling 
docker.io/lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8 not found, will build: could not find image or index for docker.io/lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8
building docker.io/lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8 for arches: arm64
Building for arch arm64 as docker.io/lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8-arm64
INFO[0000] Image docker.io/lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8 arch arm64 incomplete or invalid in local cache, error no such reference: docker.io/lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8, pulling 
No image pulled for arch arm64, continuing with build
starting existing container linuxkit-builder
waiting for buildkit builder to be ready, up to 30 seconds
buildkit builder ready!
building for platform linux/arm64
#1 [internal] load remote build context
#1 DONE 0.1s

#2 copy /context /
#2 DONE 0.0s

#3 resolve image config for docker.io/docker/buildkit-syft-scanner:stable-1
#3 DONE 0.5s

#4 [internal] load metadata for docker.io/lfedge/eve-alpine:591df01e581889c3027514c8a91feaca1c8ad49f
#4 DONE 0.3s

#5 [build-base 1/7] FROM docker.io/lfedge/eve-alpine:591df01e581889c3027514c8a91feaca1c8ad49f@sha256:911fd63beaf13cd60937b272c7c5e264321a5cdab322a7d26d29e6e3722640f9
#5 resolve docker.io/lfedge/eve-alpine:591df01e581889c3027514c8a91feaca1c8ad49f@sha256:911fd63beaf13cd60937b272c7c5e264321a5cdab322a7d26d29e6e3722640f9 0.0s done
#5 DONE 0.1s

#6 docker-image://docker.io/docker/buildkit-syft-scanner:stable-1
#6 resolve docker.io/docker/buildkit-syft-scanner:stable-1 0.1s done
#6 DONE 0.2s

#7 https://github.com/raspberrypi/firmware/raw/1.20211007/boot/start4.elf
#7 ...

#8 https://github.com/u-boot/u-boot/archive/v2022.10.tar.gz
#8 DONE 0.5s

#9 https://github.com/raspberrypi/firmware/raw/1.20211007/boot/fixup4.dat
#9 DONE 0.5s

#7 https://github.com/raspberrypi/firmware/raw/1.20211007/boot/start4.elf
#7 DONE 0.9s

#10 [build-arm64 4/5] ADD https://github.com/raspberrypi/firmware/raw/1.20211007/boot/start4.elf /tmp/rpi/start4.elf
#10 CACHED

#11 [build-arm64 3/5] ADD https://github.com/raspberrypi/firmware/raw/1.20211007/boot/fixup4.dat /tmp/rpi/fixup4.dat
#11 CACHED

#12 [build-base 3/7] ADD https://github.com/u-boot/u-boot/archive/v2022.10.tar.gz /uboot.tar.gz
#12 CACHED

#13 [build 1/6] COPY config /tmp/
#13 CACHED

#14 [build-base 5/7] WORKDIR /u-boot
#14 CACHED

#15 [build 3/6] RUN make "rpi_4_defconfig"
#15 CACHED

#16 [build-base 7/7] RUN mkdir /boot
#16 CACHED

#17 [build-arm64 2/5] COPY rpi /tmp/rpi
#17 CACHED

#18 [build-base 6/7] COPY patches /tmp/patches
#18 CACHED

#19 [build-arm64 5/5] RUN  for i in /tmp/rpi/overlays/*.dts ; do                                                   dtc -@ -I dts -O dtb -o "${i/.dts/.dtbo}" "$i" && rm "$i"     ;       done                                                                         ;       cp -r /tmp/rpi/* /boot
#19 CACHED

#20 [build 2/6] RUN cat /tmp/config >> "configs/rpi_4_defconfig"
#20 CACHED

#21 [build 5/6] RUN cp /u-boot/u-boot.bin /boot
#21 CACHED

#22 [build-base 2/7] RUN eve-alpine-deploy.sh
#22 CACHED

#23 [build 4/6] RUN make -j "$(getconf _NPROCESSORS_ONLN)"
#23 CACHED

#24 [build 6/6] RUN if [ "$(uname -m)" = aarch64 ]; then           cp /u-boot/arch/arm/dts/bcm2711-rpi-4-b.dtb /boot          ;       fi
#24 CACHED

#25 [stage-5 1/2] COPY --from=build /u-boot/u-boot* /u-boot/
#25 CACHED

#26 [build-arm64 1/5] RUN for p in /tmp/patches/patches-"v2022.10"/*.patch ; do patch -p1 < "$p" || exit 1 ; done
#26 CACHED

#27 [build-base 4/7] RUN tar -C / -xzf /uboot.tar.gz && rm -f /uboot.tar.gz && mv /u-boot* /u-boot
#27 CACHED

#28 [linux/arm64] generating sbom using docker.io/docker/buildkit-syft-scanner:stable-1
#28 CACHED

#29 [stage-5 2/2] COPY --from=build /boot /boot
#29 CACHED

#30 exporting to oci image format
#30 exporting layers done
#30 exporting manifest sha256:9f57a7fdcc9dbac152f491793677240219fc6efa97af95de239eb2c245f962d8 done
#30 exporting config sha256:b2bee71db6803f3f80c226efff9064f3c3726f95e4337c552228217d5331cbec done
#30 exporting attestation manifest sha256:31c186d64134bb75c57327e480977e52324ca0bc4b6e1c1fb894a5fd3b511f49 done
#30 exporting manifest list sha256:6d360081a66364b58afb3deac1cbb1fad9c28606ead52711501a298acedad601 done
#30 sending tarball 0.1s done
#30 DONE 0.1s
Build complete, not pushing, all done.
cd /var/home/myusername/path-to-project/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.13.50/installer/ && /var/home/myusername/path-to-project/eve/build-tools/bin/linuxkit cache export --arch arm64 --format filesystem --outfile - lfedge/eve-u-boot:0f78f2dc0eae3c44926f1c815180a8af9cbac831-dirty-e8b00e8 | tar xvf - boot
boot
boot/bcm2711-rpi-4-b.dtb
boot/config.txt
boot/fixup4.dat
boot/overlays
boot/overlays/bcm2711-spi-tpm-slb9670.dtbo
boot/overlays/raspberrypi-uno-220.dtbo
boot/start4.elf
boot/startup.nsh
boot/u-boot.bin
mkdir -p /var/home/myusername/path-to-project/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.13.50/installer/
/var/home/myusername/path-to-project/eve/build-tools/bin/linuxkit pkg build --pull --platforms linux/arm64 pkg/grub # running linuxkit pkg build _without_ force ensures that we either pull it down or build it.
Building "lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9"
checking for docker.io/lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9 in local cache...
docker.io/lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9 arm64 not found in local cache, trying to pull
INFO[0000] Image docker.io/lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9 arch arm64 incomplete or invalid in local cache, error no such reference: docker.io/lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9, pulling 
docker.io/lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9 not found, will build: could not find image or index for docker.io/lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9
building docker.io/lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9 for arches: arm64
Building for arch arm64 as docker.io/lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9-arm64
INFO[0000] Image docker.io/lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9 arch arm64 incomplete or invalid in local cache, error no such reference: docker.io/lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9, pulling 
No image pulled for arch arm64, continuing with build
using existing container linuxkit-builder
building for platform linux/arm64
#1 [internal] load remote build context
#1 DONE 0.0s

#2 copy /context /
#2 DONE 0.0s

#3 resolve image config for docker.io/docker/dockerfile-upstream:1.5.0-rc2-labs
#3 DONE 0.3s

#4 docker-image://docker.io/docker/dockerfile-upstream:1.5.0-rc2-labs@sha256:588324c9cc06f92f7a988db10a5df68e615a23d81d11a27fcdfba55220e7ba0e
#4 resolve docker.io/docker/dockerfile-upstream:1.5.0-rc2-labs@sha256:588324c9cc06f92f7a988db10a5df68e615a23d81d11a27fcdfba55220e7ba0e 0.0s done
#4 CACHED

#5 resolve image config for docker.io/docker/buildkit-syft-scanner:stable-1
#5 DONE 0.1s

#6 [internal] load metadata for docker.io/lfedge/eve-alpine:591df01e581889c3027514c8a91feaca1c8ad49f
#6 DONE 0.1s

#7 [stage-5 1/3] WORKDIR /EFI/BOOT
#7 DONE 0.0s

#8 git://git.sv.gnu.org/gnulib#d271f868a8df9bbec29049d01e056481b7a1a263
#8 DONE 0.0s

#9 [stage-5 1/3] WORKDIR /EFI/BOOT
#9 DONE 0.0s

#10 [grub-build-base 1/2] FROM docker.io/lfedge/eve-alpine:591df01e581889c3027514c8a91feaca1c8ad49f@sha256:911fd63beaf13cd60937b272c7c5e264321a5cdab322a7d26d29e6e3722640f9
#10 resolve docker.io/lfedge/eve-alpine:591df01e581889c3027514c8a91feaca1c8ad49f@sha256:911fd63beaf13cd60937b272c7c5e264321a5cdab322a7d26d29e6e3722640f9 0.0s done
#10 DONE 0.0s

#11 docker-image://docker.io/docker/buildkit-syft-scanner:stable-1
#11 resolve docker.io/docker/buildkit-syft-scanner:stable-1 0.1s done
#11 DONE 0.1s

#12 https://git.savannah.gnu.org/cgit/grub.git/snapshot/grub-2.06.tar.gz
#12 DONE 4.3s

#13 [grub-build  2/13] COPY patches-2.06  /patches-2.06
#13 CACHED

#14 [grub-build  6/13] ADD https://git.savannah.gnu.org/cgit/grub.git/snapshot/grub-2.06.tar.gz /grub.tar.gz
#14 CACHED

#15 [grub-build  4/13] COPY patches-riscv64-2.06 /patches-riscv64-2.06
#15 CACHED

#16 [stage-5 2/3] COPY --from=grub-build /grub-lib/BOOT* ./
#16 CACHED

#17 [grub-build 12/13] COPY embedded.cfg /
#17 CACHED

#18 [grub-build  1/13] COPY patches /patches
#18 CACHED

#19 copy /context /
#19 CACHED

#20 [grub-build  9/13] WORKDIR /grub
#20 CACHED

#21 [grub-build 13/13] RUN case "$(uname -m)" in   x86_64)     ./grub-mkimage -O x86_64-efi -d /grub-lib/grub/x86_64-efi -o /grub-lib/BOOTX64.EFI -p /EFI/BOOT part_gpt fat ext2 iso9660 squash4 gzio linux acpi normal cpio crypto disk boot crc64 search_disk_uuid search_part_label search_label xzio xfs video gfxterm serial gptprio chain probe reboot regexp smbios part_msdos cat echo test configfile loopback net tftp http true xen_boot efi_gop gpt gcry_sha256 measurefs efinet &&    ./grub-mkimage -O i386-pc -d /grub-lib/grub/i386-pc -o /grub-lib/BOOT.pc -p /EFI/BOOT -c /embedded.cfg part_gpt fat ext2 iso9660 squash4 gzio linux acpi normal cpio crypto disk boot crc64 search_disk_uuid search_part_label search_label xzio xfs video gfxterm serial gptprio chain probe reboot regexp smbios part_msdos cat echo test configfile loopback net tftp http true ${GRUB_MODULES_i386_pc} &&     cp /grub-lib/grub/i386-pc/boot.img /grub-lib/BOOT.img ;;  aarch64)     ./grub-mkimage -O arm64-efi -d /grub-lib/grub/arm64-efi -o /grub-lib/BOOTAA64.EFI -p /EFI/BOOT part_gpt fat ext2 iso9660 squash4 gzio linux acpi normal cpio crypto disk boot crc64 search_disk_uuid search_part_label search_label xzio xfs video gfxterm serial gptprio chain probe reboot regexp smbios part_msdos cat echo test configfile loopback net tftp http true xen_boot efi_gop gpt gcry_sha256 measurefs efinet ;;   riscv64)      ./grub-mkimage -O riscv64-efi -d /grub-lib/grub/riscv64-efi -o /grub-lib/BOOTRISCV64.EFI -p /EFI/BOOT part_gpt fat ext2 iso9660 squash4 gzio linux acpi normal cpio crypto disk boot crc64 search_disk_uuid search_part_label search_label xzio xfs video gfxterm serial gptprio chain probe reboot regexp smbios part_msdos cat echo test configfile loopback net tftp http true xen_boot efi_gop gpt gcry_sha256 measurefs efinet ;;  esac
#21 CACHED

#22 [grub-build  8/13] RUN if [ ! -d "grub" ]; then         tar -xzf /grub.tar.gz &&         mv "grub-2.06" grub &&         rm -f /grub.tar.gz &&         cd grub &&         mkdir -p /apply/patches &&         for dir in patches-2.06 patches-aarch64-2.06; do             cp -r /${dir}/* /apply/patches;         done &&         git config --global user.name "Your Name" &&         git config --global user.email "you@example.com" &&         git init . && git add . && git commit -m "init" &&         git am /apply/patches/*;     fi
#22 CACHED

#23 [grub-build 10/13] RUN ./bootstrap --gnulib-srcdir=/gnulib || ./autogen.sh
#23 CACHED

#24 [grub-build-base 2/2] RUN eve-alpine-deploy.sh
#24 CACHED

#25 [grub-build  5/13] RUN ln -s python3 /usr/bin/python &&     mkdir /grub-lib
#25 CACHED

#26 [grub-build 11/13] RUN set -e; for p in ${GRUB_PLATFORM}; do       platform=${p%%:*};       begin=$(( ${#platform} + 1 ));       opt=${p:${begin}};       if [ -f Makefile ]; then make distclean; fi;       ./configure --disable-werror --libdir=/grub-lib --with-platform="${platform}" ${opt} CFLAGS='-Os -Wno-unused-value';       make -j $(getconf _NPROCESSORS_ONLN);       make install;     done
#26 CACHED

#27 [grub-build  3/13] COPY patches-aarch64-2.06 /patches-aarch64-2.06
#27 CACHED

#28 [grub-build  7/13] ADD --keep-git-dir git://git.sv.gnu.org/gnulib#d271f868a8df9bbec29049d01e056481b7a1a263 /gnulib
#28 CACHED

#29 [stage-5 3/3] COPY rootfs.cfg grub.cfg
#29 CACHED

#30 [linux/arm64] generating sbom using docker.io/docker/buildkit-syft-scanner:stable-1
#30 CACHED

#31 exporting to oci image format
#31 exporting layers done
#31 exporting manifest sha256:44061a00f3a2d7f065a8b2f80bde7f1cc6b68c02a84c9b81a809cf3179c8c1fa done
#31 exporting config sha256:e2ed35327bf2a9e4d241f4c185b8cb7b408f6b59183fa16f0a0e706c6a298c95 done
#31 exporting attestation manifest sha256:f1e30f74bd35b12fe953ad9cc5bdbcd3ac0ad6f023ef9042784a183392b6df9b 0.0s done
#31 exporting manifest list sha256:508b477333464e6ce41cbe699988db6231787d1f4bc27a58b27ed72629071efc done
#31 sending tarball done
#31 DONE 0.0s
Build complete, not pushing, all done.
cd /var/home/myusername/path-to-project/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.13.50/installer/ && /var/home/myusername/path-to-project/eve/build-tools/bin/linuxkit cache export --arch arm64 --format filesystem --outfile - lfedge/eve-grub:29074162e1a8e800fa07e31ed9f49f0529b96803-dirty-e7ed2a9 | tar xvf - EFI
EFI
EFI/BOOT
EFI/BOOT/grub.cfg
EFI/BOOT/BOOTAA64.EFI
Done building /var/home/myusername/path-to-project/eve/build-tools/bin/linuxkit
./tools/makerootfs.sh tar  -y images/out/rootfs-xen-generic.yml -t /var/home/myusername/path-to-project/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.13.50/rootfs.tar -d /var/home/myusername/path-to-project/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.13.50/installer -a arm64
Extract kernel image: docker.io/lfedge/eve-kernel:eve-kernel-arm64-v6.1.112-generic-235a293632c7-gcc
Image docker.io/lfedge/eve-kernel:eve-kernel-arm64-v6.1.112-generic-235a293632c7-gcc arch arm64 found in local cache, not pulling
Add init containers:
Process init image: docker.io/linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
Image docker.io/linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3 arch arm64 found in local cache, not pulling
Process init image: docker.io/linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
Image docker.io/linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f arch arm64 found in local cache, not pulling
Process init image: docker.io/linuxkit/containerd:95d5f0d2d8dc63bd87e96b7b39cf026cb86125c9
Image docker.io/linuxkit/containerd:95d5f0d2d8dc63bd87e96b7b39cf026cb86125c9 arch arm64 found in local cache, not pulling
Process init image: docker.io/linuxkit/getty:e74e6cad132403d1a6d6cd25b136a7c69c99f3f7
Image docker.io/linuxkit/getty:e74e6cad132403d1a6d6cd25b136a7c69c99f3f7 arch arm64 found in local cache, not pulling
Process init image: docker.io/linuxkit/memlogd:1ded209c4cc10aa8de2099f4156164b59df14e3c
Image docker.io/linuxkit/memlogd:1ded209c4cc10aa8de2099f4156164b59df14e3c arch arm64 found in local cache, not pulling
Process init image: docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64
Image docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64 arch arm64 incomplete or invalid in local cache, error no such reference: docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64, pulling
Error: failed to build init tarball from docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: could not pull image docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: error getting manifest for trusted image docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: GET https://index.docker.io/v2/lfedge/eve-dom0-ztools/manifests/400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: MANIFEST_UNKNOWN: manifest unknown; unknown tag=400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64
2024/10/26 09:51:12 error during command execution: failed to build init tarball from docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: could not pull image docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: error getting manifest for trusted image docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: GET https://index.docker.io/v2/lfedge/eve-dom0-ztools/manifests/400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: MANIFEST_UNKNOWN: manifest unknown; unknown tag=400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64
make: *** [Makefile:696: /var/home/myusername/path-to-project/eve/dist/arm64/0.0.0-master-8e7e5004-dirty-2024-10-26.13.50/rootfs.tar] Error 1
rm images/out/rootfs-xen-generic.yml.in

Hm, so it ends again at the 

ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: error getting manifest for trusted image docker.io/lfedge/eve-dom0-ztools:400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: GET https://index.docker.io/v2/lfedge/eve-dom0-ztools/manifests/400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64: MANIFEST_UNKNOWN: manifest unknown; unknown tag=400554f321defc4d46d5a1cc71db145733ee90f7-dirty-fb6914f-arm64

I'll try reverting to the last stable release and giving you any feedback.

PhilienTaylor commented 1 month ago

So, this old version doesn't require any chmod for me, and this is with my docker set up correctly now. Moreover, it looks like the make pkg, make build-tools, and make ZARCH=arm64 HV=xen live-raw all work.

I used the five lines I put in the above text, and reverted to 12.0.4's tagged release after cloning the repo. I tried to prime the docker cache with the commands given in the five, and tried to add the Docker 12.0.4 release. (I didn't find any exactly matching?) So, based on the list in docker hub, https://hub.docker.com/r/lfedge/eve-dom0-ztools/tags?name=12.0 I ran docker pull lfedge/eve-dom0-ztools:12.0.3-lts-7-g47bfda00f-arm64

after; make pkg, make build-tools, and all seemed to work fine with no permission issues (IE: I had to chmod anything before; now I don't. )

eve/dist/arm64/12.0.4-lts$ ls
installer  live.raw  rootfs.tar
PhilienTaylor commented 1 month ago

Built fine, deployed fine. I'm going to move into the next steps of my process using this older release. :)