armv5tel-linux bootstrap tarball SIGILLs

[elitak]

Issue description

61faa4415ebd4f8b4f8c42728d7939a5162f7702 replaced the armv5tel-linux bootstrap tarball with a broken one.

Steps to reproduce

On an armv5tel-linux host, nixos-rebuild build yields:

/nix/store/994cj0fsssrdsh7ywvhy48jk8xg88igv-perl-WWW-Curl-4.17.drv
/nix/store/g6jj67a5hgxbg71n09mxmrkc88239kh0-perl-DBI-1.636.drv
/nix/store/bg54m94mf2yjgsv19a01x3lnlbs0s1zg-perl-DBD-SQLite-1.48.drv
/nix/store/c613vkz5mxnbspkv65hdnd0mqcdlimd3-libsodium-1.0.10.drv
/nix/store/y1n4kvfwm6swjwlpzjg4v6i44f0chfjv-boehm-gc-7.2f.drv
/nix/store/3xgz5px42w40d03h5r46x1v2nwnp8d24-nix-1.11.2.drv
...
building path(s) ‘/nix/store/9k1nq10mlpxid4s31zgzx79hpicdx77k-bootstrap-tools’
builder for ‘/nix/store/n62xbf683fjfszpzn9sz8cvv89whi8d9-bootstrap-tools.drv’ failed due to signal 11 (Segmentation fault)
cannot build derivation ‘/nix/store/7b8fyd7zr1w9jqd3kw94m5wr7pxzncnc-bash-4.3-p42.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/vq3p75bjjj6j4wi9fdfriqxhmwqrwwm2-bzip2-1.0.6.0.1.drv’: 1 dependencies couldn't be built

[dezgeg]

Can you try if https://github.com/dezgeg/nixpkgs/commit/30a8a494ddfd489bbaba9f24e2040e096a47bd1f fixes it?

[elitak]

No, these still segfault. Are these being cross-built by x86_64? in stdenv/linux/make-bootstrap-tools-cross.nix? I will have a look and see if I can fix it.

The ones at http://nixos-arm.dezgeg.me/bootstrap/armv5tel/ work.

[dezgeg]

Yes, both are cross-built with nix-build -A stdenv/linux/make-bootstrap-tools-cross.nix -A armv5tel.

Does manually running the standalone busybox binary from http://nixos-arm.dezgeg.me/bootstrap-2016-08-28-1409bc/armv5tel/busybox crash as well? If so, I could look at the core dump produced by running that.

[elitak]

No, busybox is fine. How can I enable enough debug info to see which binary is SIGILLing?

[dezgeg]

set -x in pkgs/stdenv/linux/scripts/unpack-bootstrap-tools.sh, I suppose.

[elitak]

Oops, it was busybox after all. I've attached the coredump that I get when manually running busybox tar using the one in https://github.com/dezgeg/nixpkgs/commit/30a8a494ddfd489bbaba9f24e2040e096a47bd1f .

core.base64.txt

[dtzWill]

Is this still an issue?

[dezgeg]

In theory there could be something that fixed it by accident.

I tried to debug the coredump at some point but the crash made no sense to me. My guess would be something to do with Musl on ARMv5, either with atomics or TLS, given that around that time the bootstrap tarball switched from using glibc to musl and that the binary works fine on v6 and v7 hardware...

[dtzWill]

Looks like this was built using musl 1.1.11, and bouncing through commit logs shortly thereafter indeed an armv5 atomics/TLS issue was fixed:

https://git.musl-libc.org/cgit/musl/commit/?id=9ee57db8834ee0d9adb1e6a84a75b0818dbfca69

Alignment issue also explains why it seems to work on qemu, which probably (?) doesn't die as badly with unaligned things. Or maybe it does and that's unrelated :). Anyway it seems likely you either already saw that commit and that's partly what informed your conclusion above, or that that commit sure seems to address the problem you describe (and has long been included in the musl used by nixpkgs).

[dezgeg]

Looks like this was built using musl 1.1.11, and bouncing through commit logs shortly thereafter indeed an armv5 atomics/TLS issue was fixed:

https://git.musl-libc.org/cgit/musl/commit/?id=9ee57db8834ee0d9adb1e6a84a75b0818dbfca69

Ah, thanks! That does sound plausible.

Alignment issue also explains why it seems to work on qemu, which probably (?) doesn't die as badly with unaligned things. Or maybe it does and that's unrelated :). Anyway it seems likely you either already saw that commit and that's partly what informed your conclusion above, or that that commit sure seems to address the problem you describe (and has long been included in the musl used by nixpkgs).

I don't have either ARMv5 hardware or QEMU set up for ARMv5, it's just that the instruction set versions are (mostly) backwards-compatible and in practice the binaries for older ARMs run fine on newer ARMs (possibly with some obsolete instructions trapped and emulated in the kernel). And to clarify the same broken v5 binary runs fine on v6/v7 so I couldn't debug it on my actual HW. I did look in musl sources once and saw it use different mechanism for various things (including fetching the TLS pointer like in the commit above) depending on the ARM ISA version, it was mostly a guess though.

[stale[bot]]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[Ericson2314]

104718 and this are very similar. This is the older one so we should probably roll that into this.

[stale[bot]]

I marked this as stale due to inactivity. → More info