search

armbian / build

Armbian Linux build framework generates custom Debian or Ubuntu image for x86, aarch64, riscv64 & armhf
https://www.armbian.com
GNU General Public License v2.0
4.26k stars 2.33k forks source link

chroot: failed to run command ‘/bin/bash’: Exec format error #5817

Closed iysheng closed 1 year ago

iysheng commented 1 year ago

What happened?

I have a laptop installed Solus OS。And I modified some build code to ignore some apt releate functions with ubuntu。Now I can build and package some deb packages with licheepi-4a as:

▸ ls output/debs/
 armbian-bsp-cli-licheepi-4a-legacy_23.11.0-trunk_riscv64__1-PC908e-V46d2-H01ba-Be827-R6632.deb
 armbian-bsp-cli-licheepi-4a_23.11.0-trunk_riscv64__1-PC908e-V46d2-H01ba-Be827-R6632.deb
 armbian-firmware_23.11.0-trunk_all__1-SAbc45-B981c-R448a.deb
 fake-ubuntu-advantage-tools_23.11.0-trunk_all__1-B0d9d-R448a.deb
 jammy
 linux-dtb-legacy-thead_23.11.0-trunk_riscv64__5.10.113-Sa76d-D0efe-P15ac-C999999Hfe66-HK01ba-Vc222-B4a24-R448a.deb
 linux-dtb-legacy-thead_23.11.0-trunk_riscv64__5.10.113-Sa76d-D0efe-P15ac-C999999Hfe66-HK01ba-Vc222-B6665-R448a.deb
 linux-headers-legacy-thead_23.11.0-trunk_riscv64__5.10.113-Sa76d-D0efe-P15ac-C999999Hfe66-HK01ba-Vc222-B4a24-R448a.deb
 linux-headers-legacy-thead_23.11.0-trunk_riscv64__5.10.113-Sa76d-D0efe-P15ac-C999999Hfe66-HK01ba-Vc222-B6665-R448a.deb
 linux-image-legacy-thead_23.11.0-trunk_riscv64__5.10.113-Sa76d-D0efe-P15ac-C999999Hfe66-HK01ba-Vc222-B4a24-R448a.deb
 linux-image-legacy-thead_23.11.0-trunk_riscv64__5.10.113-Sa76d-D0efe-P15ac-C999999Hfe66-HK01ba-Vc222-B6665-R448a.deb
 linux-u-boot-licheepi-4a-legacy_23.11.0-trunk_riscv64__2020.01-Sd82d-P6029-Ha5b8-V467b-Bd32f-R448a.deb
 linux-u-boot-licheepi-4a-legacy_23.11.0-trunk_riscv64__2020.01-Sff21-P6029-Ha5b8-V467b-Bd32f-R448a.deb
 linux-u-boot-licheepi-4a-legacy_23.11.0-trunk_riscv64__2020.01-Sff21-Pd6f6-Ha5b8-V467b-Bd32f-R448a.deb

But I stocked when pack rootfs。The error messages as below:

I wonder how could this happen? And I extrace file rootfs-riscv64-jammy-minimal_202310-624a0304d910-H01ba47-B3ce5fd.tar.zst under bash with command zstdmt -dc rootfs-riscv64-jammy-minimal_202310-624a0304d910-H01ba47-B3ce5fd.tar.zst | tar xp --xattrs -C jammy_root/ to test. When I do command with

▸ sudo chroot jammy_root/ /bin/bash echo "abc"
chroot: failed to run command ‘/bin/bash’: Exec format error

Then I file with /bin/bash in rootfs-riscv64-jammy-minimal_202310-624a0304d910-H01ba47-B3ce5fd.tar.zst as:

▸ file jammy_root/bin/bash
jammy_root/bin/bash: ELF 64-bit LSB pie executable, UCB RISC-V, RVC, double-float ABI, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-riscv64-lp64d.so.1, BuildID[sha1]=78af
774fdd526282745b82bd902a9bba4d1083a8, for GNU/Linux 4.15.0, stripped

It's build with RISC-V platform, why there need do command $(chroot "${basedir}" /bin/bash -c "apt --version | cut -d\" \" -f2") in function function create_sources_list_and_deploy_repo_key() and failed?

How to reproduce?

I do this build with command :

export SHOW_LOG=yes
export SHOW_DEBUG=yes
export SHOW_COMMAND=yes
export GHCR_MIRROR=dockerproxy
./compile.sh build UBOOT_COMPILER=riscv64-unknown-linux-gnu- KERNEL_COMPILER=riscv64-unknown-linux-gnu- GITHUB_MIRROR=ghproxy REGIONAL_MIRROR=china BOARD=licheepi-4a BRANCH=legacy BUILD_DESKTOP=no BUILD_MINIMAL=yes KERNEL_CONFIGURE=yes RELEASE=jammy

Branch

main (main development branch)

On which host OS are you observing this problem?

Other

Relevant log URL

https://paste.next.armbian.com/foferufita

Code of Conduct

github-actions[bot] commented 1 year ago

Jira ticket: AR-1896

igorpecovnik commented 1 year ago

Riscv64 support is overall not yet in the best shape but using unsupported build host is just asking for troubles. I am not familiar with how Solus is different from Debian / Ubuntu so its difficult to say, but I was not able to reproduce on officially supported Jammy build host:

./compile.sh build BOARD=licheepi-4a BRANCH=legacy BUILD_DESKTOP=no BUILD_MINIMAL=yes KERNEL_CONFIGURE=yes RELEASE=jammy SHARE_LOG=yes https://paste.next.armbian.com/rilutepeta

As I don't have hardware nor time to check, I don't know if resulted image boots.

More things are missing too:

--> (3) COMMAND: update-binfmts --enable qemu-arm bash: line 1: update-binfmts: command not found

rpardini commented 1 year ago

this is again binfmt / qemu stuff which is host-dependent.

iysheng commented 1 year ago

Riscv64 support is overall not yet in the best shape but using unsupported build host is just asking for troubles. I am not familiar with how Solus is different from Debian / Ubuntu so its difficult to say, but I was not able to reproduce on officially supported Jammy build host:

./compile.sh build BOARD=licheepi-4a BRANCH=legacy BUILD_DESKTOP=no BUILD_MINIMAL=yes KERNEL_CONFIGURE=yes RELEASE=jammy SHARE_LOG=yes https://paste.next.armbian.com/rilutepeta

As I don't have hardware nor time to check, I don't know if resulted image boots.

More things are missing too:

--> (3) COMMAND: update-binfmts --enable qemu-arm bash: line 1: update-binfmts: command not found

Thanks for your patient reply. I missed install package binfmt-support. After I installed binfmt-support package, now there have update-binfmts now.

iysheng commented 1 year ago

this is again binfmt / qemu stuff which is host-dependent.

yeah, and after I install binfmt-support from source , now I can do command with update-binfmts. But when I try do command with update-binfmts --enable qemu-arm I get these errors:

▸ sudo update-binfmts --enable qemu-riscv64
[sudo] password for yangyongsheng:
update-binfmts: warning: qemu-riscv64not in database of installed binary formats.
update-binfmts: exiting due to previous errors

After I google this, then I do command with sudo update-binfmts --install qemu-riscv64 /bin/qemu-riscv64 --magic '\x7f\x45\x4c\x46\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00' to register qemu-riscv64 support. Then After I do command sudo update-binfmts --enable qemu-riscv64, something terrible happened as:

the host OS is broken, I must repower the laptop. is there anything wrong? thanks in advance.

igorpecovnik commented 1 year ago

is there anything wrong?

Nothing is wrong with our builds tools apparently. As it works for us and in CI. We can't reproduce, we don't have resources to work on task that is officially rejected - supporting "all" Linux distributions as a build host. There are many many real support problems elsewhere ...

This can not be recognized as an issue.

I hope you accept this with understanding.

iysheng commented 1 year ago

is there anything wrong?

Nothing is wrong with our builds tools apparently. As it works for us and in CI. We can't reproduce, we don't have resources to work on task that is officially rejected - supporting "all" Linux distributions as a build host. There are many many real support problems elsewhere ...

This can not be recognized as an issue.

I hope you accept this with understanding.

Yeah, After I solve the problems with binfmt-support package, now I can build the image under Solus, but not checked 😅 . What I do just rebuild qemu with -static flag refer to https://gitlab.archlinux.org/archlinux/packaging/packages/qemu. Then I move qemu-riscv64/arm/aarch64-static to /usr/bin directory and use commands with:

sudo update-binfmts --install qemu-riscv64 /usr/bin/qemu-riscv64-static --magic '\x7f\x45\x4c\x46\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00' --mask '\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'

sudo update-binfmts --install qemu-aarch64 /usr/bin/qemu-aarch64-static --magic '\x7f\x45\x4c\x46\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00' --mask '\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'

sudo update-binfmts --install qemu-arm /usr/bin/qemu-arm-static --magic '\x7f\x45\x4c\x46\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00' --mask '\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff'

to registe these arch releate interpreter。Besides I get these magic and mask form package https://packages.debian.org/sid/qemu-user-static，a file named binfmt-install list these magic and mask. Then I use command below test success.

▸ sudo chroot . /bin/echo "abc"
[sudo] password for yangyongsheng:
abc
┏─╼ [yangyongsheng]╾╼ [14:13:06]╾╼ [0]
┗─╼ [~/LearnProjects/licheepi_4a/jammy_root]
▸ file bin/echo
bin/echo: ELF 64-bit LSB pie executable, UCB RISC-V, RVC, double-float ABI, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-riscv64-lp64d.so.1, BuildID[sha1]=3919109b52e98cf5ae25f271fc73679bc96466c8, for GNU/Linux 4.15.0, stripped

Final I rebuild licheepi 4a img, and It's success with log https://paste.next.armbian.com/esunirisuh and these images as ：

ColorfulRhino commented 7 months ago

If anyone has a seemingly similar issue on Ubuntu 23.10 or 24.04: qemu-user-static has removed binfmt-support registration from version 8.0.0 onwards (see changelog) From version 8 the qemu files for binfmt are only stored in /usr/lib/binfmt.d/

Usually, the systemd service systemd-binfmt.service will load these config files automatically on boot. On WSL though, this is prevented:

sudo systemctl status systemd-binfmt.service

○ systemd-binfmt.service - Set Up Additional Binary Formats
     Loaded: loaded (/usr/lib/systemd/system/systemd-binfmt.service; static)
    Drop-In: /etc/systemd/system/systemd-binfmt.service.d
             └─00-wsl.conf
             /usr/lib/systemd/system/systemd-binfmt.service.d
             └─wsl.conf
     Active: inactive (dead)
  Condition: start condition unmet at Fri 2024-03-29; 1h 27min ago
       Docs: man:systemd-binfmt.service(8)
             man:binfmt.d(5)
             https://docs.kernel.org/admin-guide/binfmt-misc.html
             https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems

systemd[1]: systemd-binfmt.service - Set Up Additional Binary Formats was skipped because of an unmet condition check (ConditionVirtualization=!wsl).

Temporary workaround:

Edit both files 1) /usr/lib/systemd/system/systemd-binfmt.service.d/wsl.conf 2) /etc/systemd/system/systemd-binfmt.service.d/00-wsl.conf and put the two existing lines in a comment (put a # in front of each line), like this:

#[Unit]
#ConditionVirtualization=!container

Then reboot.