libbpf / ci

BPF CI
Other
11 stars 24 forks source link

rootfs: Add riscv64 arch support #139

Closed pulehui closed 2 months ago

pulehui commented 2 months ago

@chantra Hi Manu, Hope all is well!

Currently vmtest in bpf selftests already supports riscv64 [0]. Although we also support local rootfs images, for development convenience and subsequent BPF CI support, let us support online riscv64 rootfs image.

Currently I made a commit to add the riscv64 architecture to the rootfs Makefile, but because I don’t have permission, I can’t continue to push the riscv64 rootfs image to the BPF CI Amazon server. Could you help to deal with this?

Link: https://lore.kernel.org/all/20240905081401.1894789-1-pulehui@huaweicloud.com [0]

chantra commented 2 months ago

FYI, in the case of BPF CI, we are now running the tests using https://github.com/danobi/vmtest which re-uses the local rootfs. It can also support taking a directory to take as a rootfs.

Currently, it does not support RISCV64, but with the help of https://lore.kernel.org/all/20240905081401.1894789-10-pulehui@huaweicloud.com/ it should be possible to add support.

In the meantime, I am going to build a rootfs and upload it to S3 so you can use it with the current bpf selftest script.

chantra commented 2 months ago

140 for the linking into INDEX.

chantra commented 2 months ago

@pulehui https://github.com/danobi/vmtest/pull/90 adds riscv64 support to danobi/vmtest. In my experience, it failed to boot a gzipped kernel.

Aside from me banging my head as to why qemu was not booting, the other problem is that in BPF CI, we discover the location of the kernel using something along:

$ KBUILD_OUTPUT=$KBUILD_OUTPUT_DIR make ARCH=riscv  -s image_name
arch/riscv/boot/Image.gz

which as you can see, returns the .gz image.

chantra commented 2 months ago

https://gist.github.com/chantra/05366d80873cf33379f96ff6d7c5c231 would make it so only Image is compiled and make -s image_name would return a bootable kernel.

pulehui commented 2 months ago

@chantra So much thanks for pushing riscv64 rootfs image.😁 As for the problem of failure to start qemu using Image.gz, I found that qemu [0] does not seem to support the startup of riscv64 gzipped kernel [actually I am not familiar with this]. I will try to locate this problem, but due to my recent busy work, I may not be able to reply it quickly.☹️

Link: https://gitlab.com/qemu-project/qemu/-/blob/master/hw/riscv/boot.c [0]

chantra commented 2 months ago

Thanks. For now I think the best course of action would be to include https://gist.github.com/chantra/05366d80873cf33379f96ff6d7c5c231 in the config as part of BPF CI the day there is resources to run the tests.

pulehui commented 2 months ago

Right, or convert it in the run.sh script of run-qemu:

diff --git a/run-qemu/run.sh b/run-qemu/run.sh
index 75f88ad..08c56fc 100755
--- a/run-qemu/run.sh
+++ b/run-qemu/run.sh
@@ -83,6 +83,12 @@ if [[ ${img_fs} == "tmpfs" ]]; then
    CACHE_OPT=""
 fi

+# QEMU not support riscv64 gzipped kernel yet
+if [[ ${ARCH} == "riscv64" ]] && [[ $(file $VMLINUZ) == *gzip* ]]; then
+   VMLINUZ_TMP=$VMLINUZ
+   VMLINUZ=$(mktemp) && gunzip -c $VMLINUZ_TMP > $VMLINUZ
+fi
+
 "$qemu" -nodefaults --no-reboot -nographic \
   -chardev stdio,id=char0,mux=on,signal=off,logfile=boot.log \
   -serial chardev:char0 \
@@ -90,6 +96,13 @@ fi
   -drive file="$IMG",format=raw,index=1,media=disk,if=virtio${CACHE_OPT} \
   -kernel "$VMLINUZ" -append "root=/dev/vda rw console=$console panic=-1 sysctl.vm.panic_on_oom=1 $APPEND"

+# Not sure whether tmp files will be recycled after CI completed,
+# otherwise the following steps can be omitted.
+if [[ -n $VMLINUZ_TMP ]]; then
+   rm $VMLINUZ
+   VMLINUZ=$VMLINUZ_TMP
+fi
+
 exitfile="${bpftool_exitstatus}\n"
 exitfile+="$(guestfish --ro -a "$IMG" -i cat /exitstatus 2>/dev/null)"
 exitstatus="$(echo -e "$exitfile" | awk --field-separator ':' \