search

OP-TEE / build

Makefiles to use OP-TEE on various platforms
109 stars 218 forks source link

Compilation of Rust examples fails on aarch64 hosts #752

Open msgilligan opened 5 months ago

msgilligan commented 5 months ago

Note: This is a new sub-issue from Issue #747 that addresses one of several problems that occur when trying to build on an aarch64 host.

When building (cd build; make check) on an aarch64 host with the current master branch (with a patch for Issue #749 and a workaround for Issue #751) , the following error occurs:

Compiling optee-utee-sys v0.2.0 (/home/sean.linux/optee/out-br/build/optee_rust_examples_ext-1.0/optee-utee/optee-utee-sys)
error: linking with `/home/sean.linux/optee/build/../toolchains/aarch64/bin/aarch64-linux-ld.bfd` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/bin:/home/sean.linux/optee/build/../toolchains/rust/.cargo/bin:/home/sean.linux/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/usr/sbin:/sbin:/usr/sbin:/sbin" VSLANG="1033" "/home/sean.linux/optee/build/../toolchains/aarch64/bin/aarch64-linux-ld.bfd" "/tmp/rustcte7jtz/symbols.o" "/home/sean.linux/optee/out-br/build/optee_rust_examples_ext-1.0/examples/digest-rs/ta/target/release/build/optee-utee-sys-89badbacee7da3e5/build_script_build-89badbacee7da3e5.build_script_build.7ae53caa20b5e00-cgu.0.rcgu.o" "/home/sean.linux/optee/out-br/build/optee_rust_examples_ext-1.0/examples/digest-rs/ta/target/release/build/optee-utee-sys-89badbacee7da3e5/build_script_build-89badbacee7da3e5.da5qy974ivu4kal.rcgu.o" "--as-needed" "-L" "/home/sean.linux/optee/out-br/build/optee_rust_examples_ext-1.0/examples/digest-rs/ta/target/release/deps" "-L" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-Bstatic" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libstd-cf59f2c17597b3d9.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libpanic_unwind-8f535f48df960590.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libobject-d67feb497d1e63a5.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libmemchr-70b8f4db34cf73b9.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libaddr2line-1b440c2cf263d456.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libgimli-966ed6f97fef1180.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_demangle-c5b96c1bbf2a9ac3.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libstd_detect-5732d1cd4a81a064.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libhashbrown-2a13d8dbe8a503b5.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-a747a82492cfa49f.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libminiz_oxide-11b720ff8f234cbb.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libadler-6c78e2818dc121b9.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libunwind-426a6726f3a52b5c.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcfg_if-bf93157d645f5523.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-cbccd1eea43ebbbb.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/liballoc-0b4655237a1f2f25.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/librustc_std_workspace_core-4ce1a30d69fd6755.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcore-bdfe3801ac589305.rlib" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib/libcompiler_builtins-76fca0633b54e12b.rlib" "-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "--eh-frame-hdr" "-z" "noexecstack" "-L" "/home/sean.linux/optee/toolchains/rust/.rustup/toolchains/nightly-2023-12-18-aarch64-unknown-linux-gnu/lib/rustlib/aarch64-unknown-linux-gnu/lib" "-o" "/home/sean.linux/optee/out-br/build/optee_rust_examples_ext-1.0/examples/digest-rs/ta/target/release/build/optee-utee-sys-89badbacee7da3e5/build_script_build-89badbacee7da3e5" "--gc-sections" "-pie" "-z" "relro" "-z" "now"
  = note: /home/sean.linux/optee/build/../toolchains/aarch64/bin/aarch64-linux-ld.bfd: cannot find -lgcc: No such file or directory

error: could not compile `optee-utee-sys` (build script) due to 1 previous error
warning: build failed, waiting for other jobs to finish...

The key part of the error seems to be aarch64-linux-ld.bfd: cannot find -lgcc: No such file or directory.

This comment on PR #748 indicates that this error occurs once that fix is applied. You also need to set WITH_CXX_TESTS=n to avoid Issue #751.

Update on 2024-05-24: Now that #753 and #748 are merged the above error occurs on the current master branch without needing to apply patches/workarounds for the issues addressed by those now-merged PRs.

github-actions[bot] commented 4 months ago

This issue has been marked as a stale issue because it has been open (more than) 30 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 5 days. Note, that you can always re-open a closed issue at any time.

msgilligan commented 4 months ago

Not stale!

github-actions[bot] commented 3 months ago

This issue has been marked as a stale issue because it has been open (more than) 30 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 5 days. Note, that you can always re-open a closed issue at any time.

msgilligan commented 3 months ago

Bump.

jforissier commented 3 months ago

Note having an Aarch64 machine at hand, this is difficult to investigate. Running an Aarch64 container on a x86_64 host is possible but is also quite slow especially when building a large code base. Moreover, the recently introduced arm64 runners in GitHub Actions (link) do not seem to be free for use by open source projects (link), which means a potential fix cannot be tested in CI. Therefore, contributions are welcome but do not expect a fix from the core OP-TEE contributors as we don't have the means to fix and test properly.

jforissier commented 3 months ago

Labeled as a bug because it is, and so it won't become stale and close automatically.

msgilligan commented 3 months ago

Note having an Aarch64 machine at hand, this is difficult to investigate.

I understand. (I have the mirror issue since I generally don't have an AMD64 machine at hand.)

Running an Aarch64 container on a x86_64 host is possible but is also quite slow especially when building a large code base.

What I have done is create a Containerfile that builds everything needed to compile the Rust examples and saves that as a checkpoint. You can then do a build of just the Rust examples, or even just one Rust example and you can reproduce the failed build in a few minutes. I've been using this to allow me to build just the examples with AMD64 in order to get a successful build.

If you're interested I can share it with you.

Moreover, the recently introduced arm64 runners in GitHub Actions (link) do not seem to be free for use by open source projects

😢 I've been waiting and watching.

Therefore, contributions are welcome but do not expect a fix from the core OP-TEE contributors as we don't have the means to fix and test properly.

Maybe I can get some help from the Teaclave developers. I'm not an expert on Rust linking and the link customization in the build.rs files for the Rust examples is alien to me, but if someone more experienced then me doesn't fix it soon, I will have to bite the bullet and do it myself.

jforissier commented 3 months ago

[Bad suggestion deleted]

msgilligan commented 3 months ago

[Bad suggestion deleted]

I was just about to give it a try! Darn!

Thanks for trying.