rustshop / flakebox

Flakebox is to your Rust project dev environment, what NixOS is to your OS, or home-manager to your home directory.
96 stars 5 forks source link

Cross compilation dependencies #112

Closed epigramengineer closed 8 months ago

epigramengineer commented 8 months ago

Hi! love what you're building here.

I'm using this to achieve cross compilation from my x86 to a raspberry pi 4 (aarch64). While building works great, when I copy over the executable to my raspberry pi (also running nixos) and run it I get no such file or directory which indicates a problem with linking (according to my google searching).

Here is the file of the output:

❯ file result/bin/solver
result/bin/solver: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /nix/store/5gdh4mp8rwliq4s33gwcpwzqvsb2xpzr-glibc-aarch64-unknown-linux-gnu-2.38-23/lib/ld-linux-aarch64.so.1, for GNU/Linux 3.10.0, with debug_info, not stripped

And running ldd on the raspberry pi:

❯ ldd solver 
    linux-vdso.so.1 (0x0000007fb649c000)
    libc.so.6 => /nix/store/ilr14j6gzr9l651yb5w367y00cs6pflb-glibc-2.37-45/lib/libc.so.6 (0x0000007fb5ae0000)
    libm.so.6 => /nix/store/ilr14j6gzr9l651yb5w367y00cs6pflb-glibc-2.37-45/lib/libm.so.6 (0x0000007fb5a30000)
    libgcc_s.so.1 => /nix/store/x2f8wiip66lxf6zynscszpm4d990q098-xgcc-12.2.0-libgcc/lib/libgcc_s.so.1 (0x0000007fb59f0000)
    /nix/store/5gdh4mp8rwliq4s33gwcpwzqvsb2xpzr-glibc-aarch64-unknown-linux-gnu-2.38-23/lib/ld-linux-aarch64.so.1 => /nix/store/ilr14j6gzr9l651yb5w367y00cs6pflb-glibc-2.37-45/lib/ld-linux-aarch64.so.1 (0x0000007fb645f000)
❯ objdump -j .interp -s ./solver

./solver:     file format elf64-littleaarch64

Contents of section .interp:
 0270 2f6e6978 2f73746f 72652f35 67646834  /nix/store/5gdh4
 0280 6d703872 776c6971 34733333 67776370  mp8rwliq4s33gwcp
 0290 777a7176 73623278 707a722d 676c6962  wzqvsb2xpzr-glib
 02a0 632d6161 72636836 342d756e 6b6e6f77  c-aarch64-unknow
 02b0 6e2d6c69 6e75782d 676e752d 322e3338  n-linux-gnu-2.38
 02c0 2d32332f 6c69622f 6c642d6c 696e7578  -23/lib/ld-linux
 02d0 2d616172 63683634 2e736f2e 3100      -aarch64.so.1.  

While none of the dependencies are missing, it appears the interpreter is set to /nix/store/5gdh4mp8rwliq4s33gwcpwzqvsb2xpzr-glibc-aarch64-unknown-linux-gnu-2.38-23/lib/ld-linux-aarch64.so.1 (which is on my x86 system), and while ldd states this should link to /nix/store/ilr14j6gzr9l651yb5w367y00cs6pflb-glibc-2.37-45/lib/ld-linux-aarch64.so.1 it doesn't appear to do that when it runs.. Honestly I'm a bit out of my depth on this.

I'm wondering if anyone else has run into this and how it was resolved? It looks like statically linking is one option, but that is only supported by musl not glibc, which flakebox doesn't appear to support as the target is set to:

  aarch64-linux = mkClangToolchain {
    target = "aarch64-unknown-linux-gnu";
    ...

I suppose another option would be to ensure that /nix/store/5gdh4mp8rwliq4s33gwcpwzqvsb2xpzr-glibc-aarch64-unknown-linux-gnu-2.38-23/lib/ld-linux-aarch64.so.1 is indeed present on the raspberry pi system, but I'm not really sure how to do that on a continuous basis, I don't want to have to rebuild the system when I deploy my rust application to it

I also see that the intepreter is set to gnu-2.38-23 but links to 2.37-45 but if it were just a version change, why would it link to that in the first place?

epigramengineer commented 8 months ago

Ah; I found https://discourse.nixos.org/t/cross-compilation-for-raspberry-pi-4b/11163 which suggests using nix-copy-closure which works perfectly