qemu_v8: Enable Rust examples build by default

[b49020]

With no-std support it has significantly reduced time to build and test rust examples. So enable rust examples build by default for qemu_v8. Along with that add rust expect script to build repo to avoid check script duplication in OP-TEE rust SDK repo.

@DemesneGH Fyi..

[b49020]

IBART failure seems unrelated to this PR, @jbech-linaro please have a look.

[b49020]

Addressed review comments, thanks.

[DemesneGH]

Acked-by: Yuan Zhuang <yuanz@apache.org>

[b49020]

Tags applied, thanks.

[jforissier]

@b49020 the QEMUv8 make COMPILE_S_USER=32 fails with:

     Running `/home/jerome/work/optee_repo_qemu_v8/out-br/build/optee_rust_examples_ext-1.0/examples/acipher-rs/ta/target/release/build/ta-3b25c1a88446c8a1/build-script-build`
error: could not find native static library `utee`, perhaps an -L flag is missing?

error: could not compile `optee-utee-sys` due to previous error

Caused by:
  process didn't exit successfully: `rustc --crate-name optee_utee_sys --edition=2018 /home/jerome/work/optee_repo_qemu_v8/out-br/build/optee_rust_examples_ext-1.0/optee-utee/optee-utee-sys/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=1 -C linker-plugin-lto -C metadata=d4ddde63eaa75a7a -C extra-filename=-d4ddde63eaa75a7a --out-dir /home/jerome/work/optee_repo_qemu_v8/out-br/build/optee_rust_examples_ext-1.0/examples/acipher-rs/ta/target/aarch64-unknown-optee-trustzone/release/deps --target aarch64-unknown-optee-trustzone -C linker=aarch64-linux-gnu-ld -L dependency=/home/jerome/work/optee_repo_qemu_v8/out-br/build/optee_rust_examples_ext-1.0/examples/acipher-rs/ta/target/aarch64-unknown-optee-trustzone/release/deps -L dependency=/home/jerome/work/optee_repo_qemu_v8/out-br/build/optee_rust_examples_ext-1.0/examples/acipher-rs/ta/target/release/deps --extern libc=/home/jerome/work/optee_repo_qemu_v8/out-br/build/optee_rust_examples_ext-1.0/examples/acipher-rs/ta/target/aarch64-unknown-optee-trustzone/release/deps/liblibc-ed330a60b78d74ed.rmeta -C link-arg=-e__ta_entry -C link-arg=-nostdlib -C link-arg=-Tta.lds -C link-arg=--sort-section=alignment -C link-arg=-pie -C link-arg=-Os -C link-dead-code --sysroot /home/jerome/.xargo -L /home/jerome/work/optee_repo_qemu_v8/out-br/build/optee_rust_examples_ext-1.0/../../../optee_os/out/arm/export-ta_arm64/lib -l static=utee -l static=mbedtls` (exit status: 1)
warning: build failed, waiting for other jobs to finish...
error: build failed

Should OPTEE_RUST_ENABLE be set to y only for builds with 64-bit TAs?

[b49020]

@jforissier Rust supports 32-bit TAs [1] too but I have to make them work with OP-TEE build system too. I will add corresponding support prior to change in the PR.

[1] https://github.com/apache/incubator-teaclave-trustzone-sdk/blob/no-std/.github/workflows/ci.yml#L90

[jforissier]

@b49020 excellent, thanks!

[b49020]

@jforissier Now make COMPILE_S_USER=32 should pass on Qemuv8. Please have a look at the first patch.

[jforissier]

There is a OPTEE_RUST_ENABLE left over in common.mk. With that fixed and the following lines added to the Ubuntu 22.04 Dockerfile in https://optee.readthedocs.io/en/latest/building/prerequisites.html:

RUN curl -sSf https://sh.rustup.rs | sh -s -- -y
ENV PATH="$PATH:/root/.cargo/bin"
RUN rustup target add aarch64-unknown-linux-gnu arm-unknown-linux-gnueabihf

...the make -j$(nproc) check CHECK_TESTS=rust command passes.

However, make -j$(nproc) check CHECK_TESTS=rust COMPILE_S_USER=32 fails with:

[...]
buildroot login: root
# export LD_LIBRARY_PATH=/lib:/lib/arm-linux-gnueabihf
# acipher-rs 256 teststring
Error: Input data was of invalid format. (error code 0xffff0005)
# == end of /home/builder/optee/out/bin/serial0.log

[b49020]

There is a OPTEE_RUST_ENABLE left over in common.mk.

Ah I missed that, it is the reason for your build and test issues as https://github.com/apache/incubator-teaclave-trustzone-sdk/blob/no-std/setup.sh isn't executed to install specific rust nightly toolchain and dependencies (cargo etc.).

RUN curl -sSf https://sh.rustup.rs | sh -s -- -y ENV PATH="$PATH:/root/.cargo/bin" RUN rustup target add aarch64-unknown-linux-gnu arm-unknown-linux-gnueabihf

This won't be needed once setup.sh executes.

However, make -j$(nproc) check CHECK_TESTS=rust COMPILE_S_USER=32 fails with:

Can you retry this again as I have fixed OPTEE_RUST_ENABLE left over as we depend on a particular Rust compiler nightly release: https://github.com/apache/incubator-teaclave-trustzone-sdk/blob/no-std/rust-toolchain.toml?

[b49020]

Addressed comments.

[b49020]

However, make -j$(nproc) check CHECK_TESTS=rust COMPILE_S_USER=32 fails with:

Can you retry this again as I have fixed OPTEE_RUST_ENABLE left over as we depend on a particular Rust compiler nightly release: https://github.com/apache/incubator-teaclave-trustzone-sdk/blob/no-std/rust-toolchain.toml?

Also, please clean up the prior build too via:

rm -rf ../out-br/build/optee_rust_examples_ext-1.0/

[jforissier]

$ make -j10 run COMPILE_S_USER=32
[...]
Welcome to Buildroot, type root or test to login
buildroot login: test
$ acipher-rs 256 teststring
Error: Input data was of invalid format. (error code 0xffff0005)

D/TC:? 0 ldelf_syscall_open_bin:163 Lookup user TA ELF 057f4b66-bdab-11eb-96cf-33d6e41cc849 (REE)
D/TC:? 0 ldelf_syscall_open_bin:167 res=0
E/LD:  init_elf:511 Cannot parse ELF
E/TC:? 0 ldelf_init_with_ldelf:152 ldelf failed with res: 0xffff0005
D/TC:? 0 tee_ta_open_session:689 init session failed 0xffff0005

It looks like the TA ELF file has something that the OP-TEE loader doesn't like.

[b49020]

It looks like the TA ELF file has something that the OP-TEE loader doesn't like.

Ah, I can reproduce this issue now. Somehow it passed earlier because my build environment wasn't cleaned properly. I will dig into this issue.

[jforissier]

I can see at least two fields in the ELF header that are unexpected. OP-TEE should probably accept "OS/ABI: ARM", but I'm not so sure about flags being zero.

build$ arm-linux-gnueabihf-readelf -h ../out-br/build/optee_rust_examples_ext-1.0/examples/acipher-rs/ta/target/arm-unknown-linux-gnueabihf/release/stripped_ta ELF Header: Magic: 7f 45 4c 46 01 01 01 61 00 00 00 00 00 00 00 00 Class: ELF32 Data: 2's complement, little endian Version: 1 (current) OS/ABI: ARM ABI Version: 0 Type: DYN (Position-Independent Executable file) Machine: ARM Version: 0x1 Entry point address: 0x3340 Start of program headers: 52 (bytes into file) Start of section headers: 145660 (bytes into file) Flags: 0x0 Size of this header: 52 (bytes) Size of program headers: 32 (bytes) Number of program headers: 4 Size of section headers: 40 (bytes) Number of section headers: 19 Section header string table index: 18

build$ $ arm-linux-gnueabihf-readelf -h ./build/optee_examples_ext-1.0/acipher/ta/out/a734eed9-d6a1-4244-aa50-7c99719e7b7b.stripped.elf ELF Header: Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 Class: ELF32 Data: 2's complement, little endian Version: 1 (current) OS/ABI: UNIX - System V ABI Version: 0 Type: DYN (Position-Independent Executable file) Machine: ARM Version: 0x1 Entry point address: 0x28c Start of program headers: 52 (bytes into file) Start of section headers: 70944 (bytes into file) Flags: 0x5000400, Version5 EABI, hard-float ABI Size of this header: 52 (bytes) Size of program headers: 32 (bytes) Number of program headers: 5 Size of section headers: 40 (bytes) Number of section headers: 19 Section header string table index: 18

[b49020]

@jforissier

I can see at least two fields in the ELF header that are unexpected. OP-TEE should probably accept "OS/ABI: ARM", but I'm not so sure about flags being zero.

As per ELF manpage [1], the flags field value is dependent on EI_OSABI which is essentially EF_ARM_EABI_UNKNOWN in case of ELFOSABI_ARM. Also, I have tried different ways to configure Rust 32-bit TAs in no-std mode but I still see the same OS ABI being used. Hence, I have come up with this PR: https://github.com/OP-TEE/optee_os/pull/6605, please have a look.

[1] https://man7.org/linux/man-pages/man5/elf.5.html

[jforissier]

With https://github.com/OP-TEE/optee_os/pull/6605 the TA panics.

E/TC:? 0 User mode prefetch-abort at address 0x40038340 (read permission fault)
E/TC:? 0  esr 0x8200000f  ttbr0 0x400000e1b2000   ttbr1 0x00000000   cidr 0x0
E/TC:? 0  cpu #0          cpsr 0x00000110
E/TC:? 0  x0  0000000000000000 x1  000000000009d870
E/TC:? 0  x2  0000000040015f80 x3  0000000000000000
E/TC:? 0  x4  0000000000000000 x5  0000000000000000
E/TC:? 0  x6  0000000000000000 x7  0000000000000000
E/TC:? 0  x8  0000000000000000 x9  0000000000000000
E/TC:? 0  x10 0000000000000000 x11 0000000000000000
E/TC:? 0  x12 0000000000000000 x13 0000000040015f80
E/TC:? 0  x14 0000000000000000 x15 0000000000000000
E/TC:? 0  x16 0000000000000000 x17 0000000000000000
E/TC:? 0  x18 0000000000000000 x19 0000000000000000
E/TC:? 0  x20 0000000000000000 x21 0000000000000000
E/TC:? 0  x22 0000000000000000 x23 0000000000000000
E/TC:? 0  x24 0000000000000000 x25 0000000000000000
E/TC:? 0  x26 0000000000000000 x27 0000000000000000
E/TC:? 0  x28 0000000000000000 x29 0000000000000000
E/TC:? 0  x30 0000000000000000 elr 0000000040038340
E/TC:? 0  sp_el0 0000000040015f80
E/LD:  Status of TA 057f4b66-bdab-11eb-96cf-33d6e41cc849
E/LD:   arch: arm
E/LD:  region  0: va 0x40005000 pa 0x0e32c000 size 0x002000 flags rw-s (ldelf)
E/LD:  region  1: va 0x40007000 pa 0x0e32e000 size 0x008000 flags r-xs (ldelf)
E/LD:  region  2: va 0x4000f000 pa 0x0e336000 size 0x001000 flags rw-s (ldelf)
E/LD:  region  3: va 0x40010000 pa 0x0e337000 size 0x004000 flags rw-s (ldelf)
E/LD:  region  4: va 0x40014000 pa 0x0e33b000 size 0x001000 flags r--s
E/LD:  region  5: va 0x40015000 pa 0x0e35c000 size 0x001000 flags rw-s (stack)
E/LD:  region  6: va 0x40035000 pa 0x00010000 size 0x020000 flags rwxs [0]
E/LD:   [0] 057f4b66-bdab-11eb-96cf-33d6e41cc849 @ 0x40035000
E/LD:  Call stack:
E/LD:   0x40038340
E/LD:   0x00000000
D/TC:? 0 user_ta_enter:201 tee_user_ta_enter: TA panicked with code 0xdeadbeef
D/TC:? 0 release_ta_ctx:663 Releasing panicked TA ctx
D/TC:? 0 tee_ta_close_session:463 csess 0x9b91f860 id 2
D/TC:? 0 tee_ta_close_session:482 Destroy session
D/TC:? 0 destroy_context:321 Destroy TA ctx (0x9b91f800)
E/TC:? 0 tee_ta_open_session:738 Failed for TA 057f4b66-bdab-11eb-96cf-33d6e41cc849. Return error 0xffff3024

The Rust environment doesn't produce a file (or symlink) called 057f4b66-bdab-11eb-96cf-33d6e41cc849.elf so it is not convenient to run symbolize.py. Anyways, the rwxs permissions for region 6 look suspicious (writable and executable at the same time is forbidden by some config flag I think?)

[b49020]

The Rust environment doesn't produce a file (or symlink) called 057f4b66-bdab-11eb-96cf-33d6e41cc849.elf so it is not convenient to run symbolize.py.

I hope debugging Rust TAs situation would improve further in future since we also have to map symbols to Rust code as well. However, the first step that I am trying to take here is enable 32-bit Rust TAs to run.

Anyways, the rwxs permissions for region 6 look suspicious (writable and executable at the same time is forbidden by some config flag I think?)

Thanks for catching that, the fix is already available here: https://github.com/apache/incubator-teaclave-trustzone-sdk/pull/118 to be merged.

[b49020]

Addressed comments and tags applied, thanks.