Figure out how to deal with `cargo test` with a no_std target

[phip1611]

@ehuss: This is just a general issue on figuring out what cargo test should do with a no_std target.

---

This bug originates from a stack overflow thread in that my minimal project, that is building an application for the x86_64-unknown-uefi target, cannot execute cargo test. The crate itself uses the build-std feature in .config/cargo.toml. When tests are executed, the compiler complains about can't find crate for 'test'. If I add test to the build-std-array, therefore,

[unstable]
build-std = [
    "alloc",
    "compiler_builtins",
    "core",
    + "test",
]

the compiler tells I should add #![feature(restricted_std)]. If I do so, nothing changes. I found the bug with Rust/Cargo 1.55-nightly

Expected Behaviour One should be able to execute tests when build-std is used.

[ehuss]

Thanks for the report! When filing an issue about an error, it can be helpful to include the entire output, as it can include important information.

In this case, I think it is failing while building term. I have a suspicion this particular error will be fixed by https://github.com/rust-lang/rust/pull/87247, but I'm not sure.

In general, running tests in a no_std environment may not work at all. Tests require threads, allocation, and other things that may not be available. I don't recall what x86_64-unknown-uefi supports, but I suspect tests in general won't work on that target.

[hellow554]

https://github.com/rust-lang/wg-cargo-std-aware/issues/69 could be related as well

[ehuss]

Transferred this to wg-cargo-std-aware for tracking purposes.

[jschwe]

the compiler tells I should add #![feature(restricted_std)] . If I do so, nothing changes. I found the bug with Rust/Cargo 1.55-nightly

I ran into the same problem. If you add #![feature(restricted_std)] to the test crate (and not your own crate), then everything compiles fine though. Can we just add the feature to the test crate? I have no idea what it does. The documentation is not helpful.

[jyn514]

Here is the error output from building test for x86_64-unknown-uefi:

error[E0658]: use of unstable library feature 'restricted_std'
  |
  = help: add `#![feature(restricted_std)]` to the crate attributes to enable

error[E0658]: use of unstable library feature 'restricted_std'
   --> /home/jyn/.local/lib/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:126:25
    |
126 |                         std::mem::forget(std::io::stderr().lock());
    |                         ^^^^^^^^^^^^^^^^
    |
    = help: add `#![feature(restricted_std)]` to the crate attributes to enable

...

error: could not compile `test` due to 11 previous errors

but adding the feature doesn't actually help here, because cargo doesn't know to build panic_abort:

   Compiling example v0.1.0 (/home/jyn/src/example)
error[E0463]: can't find crate for `panic_abort`

when I pass -Zbuild-std=core,alloc,std,test,panic_abort I got

error: test failed, to rerun pass `--lib`

Caused by:
  could not execute process `/home/jyn/.local/lib/cargo/target/x86_64-unknown-uefi/debug/deps/example-ab333d74cf12b661.efi` (never executed)

Caused by:
  Exec format error (os error 8)

presumably because cargo is trying to execute these natively instead of through QEMU. If I use QEMU to run the binary it ends up ... hanging forever, probably because something is panicking.

So just adding restricted-std doesn't seem like it's a fix, and I'm not sure it makes sense to land without other changes to cargo or libtest; the first step here would be to investigate the panic when using QEMU to run the test binary.

commands I had to install

``` cargo install uefi-run sudo apt install qemu-system-x86 rustup component add llvm-tools # stage1 is so I use the fork of std in https://github.com/rust-lang/rust/pull/97037 # this command will error; execute the command that fails in QEMU. PATH=$PATH:/home/jyn/.local/lib/rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin rustup run stage1 cargo test -Zbuild-std=core,alloc,std,test,panic_abort --target x86_64-unknown-uefi uefi-run /home/jyn/.local/lib/cargo/target/x86_64-unknown-uefi/debug/deps/example-ab333d74cf12b661.efi` ```

[jschwe]

So just adding restricted-std doesn't seem like it's a fix, and I'm not sure it makes sense to land without other changes to cargo or libtest; the first step here would be to investigate the panic when using QEMU to run the test binary.

I've made a short (working) example of what I have in mind based on blog_os post #04

This can be run with a nightly toolchain with the llvm-preview-tools and rust-src installed with cargo test --test '*' ( I only changed the integration tests).

Additionally, the rust source code must be edited to include this patch. The compiler doesn't need to be rebuilt since build-std will rebuild test for us. alloc is necessary to compile, but doesn't need to be working, as long as the user doesn't construct anything that allocates. Custom panic handlers don't work since std is pulled in, but presumably we could set a panic-hook (haven't tested this yet).

Anyway, this allows us to successfully build test and use custom_test_frameworks to define our own test runner, that can still access all the nice information that #[test] stores in the TestDescAndFn struct (e.g. the test name, should-panic, ignore, etc.) .

[bjorn3]

Anyway, this allows us to successfully build test and use custom_test_frameworks to define our own test runner, that can still access all the nice information that #[test] stores in the TestDescAndFn struct (e.g. the test name, should-panic, ignore, etc.) .

You could also use a proc macro to define your own #[test] which adds a #[test_case] to a static storing your own TestDescAndFn type rather than directly to the function. This is effectively what libtest does, except instead of an external proc macro the #[test] attribute is built into the macro expander of rustc.

[jschwe]

You could also use a proc macro to define your own #[test] which adds a #[test_case] to a static storing your own TestDescAndFn type rather than directly to the function.

It's probably possibly, but it seems to be a bit tricky to add test cases across mod/file boundaries to the static though.

[bjorn3]

I mean having a different static for every test, not a single static for the whole crate. So for example #[my_test] fn foo() {} mod bar { #[my_test] #[my_ignore] fn baz() {} } would turn into

#[test_case]
static FOO: TestDescAndFn = TestDescAndFn {
    func: foo,
    ignore: false,
}

fn foo() {}

mod bar {
    #[test_case]
    static BAZ: TestDescAndFn = TestDescAndFn {
        func: baz,
        ignore: true,
    }

    fn baz() {}
}

All items annotated with #[test_case] are passed to the test runner. They can't just be functions, but also statics. The only requirement is that all of them have the same type.

[jschwe]

Ah, thanks that makes sense! I'll be sure to try this out once I get the chance.

[brainstorm]

I think that figuring out this issue is also worthwhile for embedded target simulators (perhaps via runners?).

See https://github.com/Patryk27/avr-tester/issues/5 where I'm trying to use plain cargo [build|test|run] to build, flash and test my Rust AVR target example application... both build and flash work as intended, but unfortunately cargo test which should launch simavr is tricky due to the different targets interplay.