Solve arm64 kraft errors

[razvanvirtan]

This PR comes to solve issue https://github.com/unikraft/kraft/issues/57. There are two problems with kraft on ARM64, solved by two different commits:

• kraft build uses aarch64-linux-gnu-gcc, which in turn calls the x86_64 assembler, resulting in a build error. At first, I was thinking about replacing the current arm64 toolchain with the linaro one (as mentioned in a previous comment). Finally, I managed to solve this by adding a missing directory (aarch64-linux-gnu) to the final kraft container. This way, we don't need to add linaro anymore. In order for this to work fine, I think we should also update the unikraft/gcc:9.2.0-arm64-staging image from docker hub[1].

• kraft run is always trying to enable hardware acceleration, even if the host architecture is different from the emulated one. This generates errors when trying to emulate ARM64 images on x86 hosts. The problem can be easily solved by checking the architecture in kvm.py and generating the right qemu command.

[1]https://hub.docker.com/layers/unikraft/gcc/9.2.0-arm64/images/sha256-3fd1ce6b6f7acb9e259212bf922b8b2090f10cf0e93621f94f04c51a92a5835e?context=explore

Signed-off-by: Răzvan Vîrtan virtanrazvan@gmail.com

[nderjung]

Hey @razvanvirtan!

Thanks for taking the time to look into this, really appreciated!

Looks like the dockerfile can't be built on the gcc image:

#10 [gcc 5/5] COPY --from=gcc-build /out/aarch64-linux-gnu/ /aarch64-linux-gnu/
#10 sha256:6b47cccdb8e29e627f2f5a463965f290687ca5dd1e07107b9b531f339326a5ec
#10 ERROR: "/out/aarch64-linux-gnu" not found: not found
------
 > [gcc 5/5] COPY --from=gcc-build /out/aarch64-linux-gnu/ /aarch64-linux-gnu/:
------
Dockerfile.gcc:146
--------------------
 144 |     COPY --from=gcc-build /out/lib/ /lib/
 145 |     COPY --from=gcc-build /out/libexec/ /libexec/
 146 | >>> COPY --from=gcc-build /out/aarch64-linux-gnu/ /aarch64-linux-gnu/
 147 |     
--------------------
error: failed to solve: rpc error: code = Unknown desc = failed to compute cache key: "/out/aarch64-linux-gnu" not found: not found
FATA[1561] failed to build: build: exit status 1        
FATA[1561] failed to run task: exit status 1            

[razvanvirtan]

Hey @razvanvirtan!

Thanks for taking the time to look into this, really appreciated!

Looks like the dockerfile can't be built on the gcc image:

#10 [gcc 5/5] COPY --from=gcc-build /out/aarch64-linux-gnu/ /aarch64-linux-gnu/
#10 sha256:6b47cccdb8e29e627f2f5a463965f290687ca5dd1e07107b9b531f339326a5ec
#10 ERROR: "/out/aarch64-linux-gnu" not found: not found
------
 > [gcc 5/5] COPY --from=gcc-build /out/aarch64-linux-gnu/ /aarch64-linux-gnu/:
------
Dockerfile.gcc:146
--------------------
 144 |     COPY --from=gcc-build /out/lib/ /lib/
 145 |     COPY --from=gcc-build /out/libexec/ /libexec/
 146 | >>> COPY --from=gcc-build /out/aarch64-linux-gnu/ /aarch64-linux-gnu/
 147 |     
--------------------
error: failed to solve: rpc error: code = Unknown desc = failed to compute cache key: "/out/aarch64-linux-gnu" not found: not found
FATA[1561] failed to build: build: exit status 1        
FATA[1561] failed to run task: exit status 1            

I think I know where the issue comes from, will come with a third commit after testing it.

[razvanvirtan]

@nderjung The gcc images can be built now. However, we still have problems with the kraft one. Without seeing the check output, my guess is that the system tries to build the kraft image using gcc images from dockerhub: https://github.com/unikraft/kraft/blob/29c6fba81b885d5fb3b5d2dc76b976644828df30/package/docker/Dockerfile.kraft#L36

Since those images have been built using the initial Dockerfile.gcc, they don't contain the directories added in this PR, so the kraft container build fails when it tries to copy x86_64-linux-gnu. I think we will need to update the gcc images from dockerhub before this to work. (I can build the kraft container on my local, if I'm using gcc images compiled with the updated Dockerfile.gcc).

[nderjung]

Hey, apologies for not being able to see the CI output yet, we're still migrating services...

Here's the relevant log:

#24 [kraft 14/31] COPY --from=gcc-arm64 /aarch64-linux-gnu/ /aarch64-linux-gnu
#24 sha256:4fddea67477d34a2ce98cb9f68cb4ad316dafd1e6196132997b1e2a28e08767f
#24 CANCELED
------
 > [kraft 10/31] COPY --from=gcc-arm /arm-linux-gnueabihf  /arm-linux-gnueabihf:
------
Dockerfile.kraft:56
--------------------
  54 |     COPY --from=gcc-arm /lib/gcc/             /lib/gcc
  55 |     COPY --from=gcc-arm /libexec/gcc/         /libexec/gcc
  56 | >>> COPY --from=gcc-arm /arm-linux-gnueabihf  /arm-linux-gnueabihf
  57 |     
  58 |     
--------------------
error: failed to solve: rpc error: code = Unknown desc = failed to compute cache key: "/arm-linux-gnueabihf" not found: not found

[razvanvirtan]

Hey, apologies for not being able to see the CI output yet, we're still migrating services...

Here's the relevant log:

#24 [kraft 14/31] COPY --from=gcc-arm64 /aarch64-linux-gnu/ /aarch64-linux-gnu
#24 sha256:4fddea67477d34a2ce98cb9f68cb4ad316dafd1e6196132997b1e2a28e08767f
#24 CANCELED
------
 > [kraft 10/31] COPY --from=gcc-arm /arm-linux-gnueabihf  /arm-linux-gnueabihf:
------
Dockerfile.kraft:56
--------------------
  54 |     COPY --from=gcc-arm /lib/gcc/             /lib/gcc
  55 |     COPY --from=gcc-arm /libexec/gcc/         /libexec/gcc
  56 | >>> COPY --from=gcc-arm /arm-linux-gnueabihf  /arm-linux-gnueabihf
  57 |     
  58 |     
--------------------
error: failed to solve: rpc error: code = Unknown desc = failed to compute cache key: "/arm-linux-gnueabihf" not found: not found

Thanks! Seems to confirm my initial thoughts. We could update these images:

• https://hub.docker.com/layers/unikraft/gcc/9.2.0-arm64/images/sha256-3fd1ce6b6f7acb9e259212bf922b8b2090f10cf0e93621f94f04c51a92a5835e?context=explore
• https://hub.docker.com/layers/unikraft/gcc/9.2.0-arm/images/sha256-234c6dcd255a04149420e0d52ca3ee094ae32a6c9af845bc699a904e0b025f23?context=explore
• https://hub.docker.com/layers/unikraft/gcc/9.2.0-x86_64/images/sha256-49b686ad57d76be519b0a0b6313d843abe3c1e490a1ba240a5efb30e088e96e4?context=explore