rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
98.13k stars 12.69k forks source link

Tracking Issue for Cross Compiling Doctests #64245

Open Goirad opened 5 years ago

Goirad commented 5 years ago

This is an issue to track the functionality added by PR#60387, and also by extension this PR in cargo.

PR#60387 adds three options to rustdoc:

The companion PR adds a flag to cargo for enabling this feature as well as cross-compiling doctests and parsing runtool tokens from a .cargo/config.

Eventually another PR for x.py can either enable this unconditionally or add a flag for testing the extensive documentation in the standard library on other platforms.

This solves cargo issue #6460

jyn514 commented 3 years ago

@Goirad I don't quite understand - what is this tracking? It sounds like all the functionality is already implemented in rustdoc, what's left to do?

ehuss commented 2 years ago

@jyn514 The rustdoc flags are still unstable, so this is tracking the stabilization of those flags.

safinaskar commented 2 years ago

My host is x86_64-unknown-linux-gnu. I'm building for x86_64-unknown-linux-musl. Resulting binaries can be run on this host without any problems. But cargo test --target=x86_64-unknown-linux-musl -Zdoctest-xcompile simply compiles test instead of running them. Moreover, this command doesn't even warn that doctests was merely compiled instead of run! I want tests to run. And I don't want some kind of Cargo.toml configuring for this.

Moreover, if I don't add -Zdoctest-xcompile, doctests are not compiled at all, and there is no any warning about this

ehuss commented 2 years ago

@safinaskar Can you provide some more information? The tests appear to be run for me with cross compiling to x86_64-unknown-linux-musl:

~/Temp/foo> cargo test --target x86_64-unknown-linux-musl -Zdoctest-xcompile
    Finished test [unoptimized + debuginfo] target(s) in 0.00s
     Running unittests (target/x86_64-unknown-linux-musl/debug/deps/foo-65396ef18f7af6ca)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests foo

running 1 test
test src/lib.rs - abc (line 1) ... FAILED

failures:

---- src/lib.rs - abc (line 1) stdout ----
Test executable failed (exit code 101).

stderr:
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `1`,
 right: `2`', src/lib.rs:3:1
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

As for warnings, there is a warning displayed when the verbose flag:

~/Temp/foo> cargo test --target x86_64-unknown-linux-musl --doc -v
       Fresh foo v0.1.0 (/home/eric/Temp/foo)
    Finished test [unoptimized + debuginfo] target(s) in 0.00s
note: skipping doctests for foo v0.1.0 (/home/eric/Temp/foo) (lib), cross-compilation doctests are not yet supported
See https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#doctest-xcompile for more information.

The reason it is only displayed with -v is that otherwise it would be displayed 100% of the time when cross-compiling, and there is no workaround other than using nightly. We felt that would be an over-agressive warning for now.

We definitely recognize that can cause people to not know that tests are not being run. Hopefully most of the time that is evident by the absence of any doc-test display.

I would like to see this feature move towards stabilization to help prevent that oversight, so any experience reports would be helpful if you get it to work.

safinaskar commented 2 years ago

@ehuss

Can you provide some more information?

I am unable to reproduce. I. e. now everything works. It seems there was no problem in the first place, sorry

kpfleming commented 1 month ago

For what it's worth, we've been using this unstable feature at Fastly for most of this year (2024), compiling a moderate number of code examples during every CI run for one of our documentation repositories (the target is wasm32-wasi). We haven't had any issues with it, it works well for us.

ehuss commented 1 month ago

Just an update, I had a small discussion about this at https://rust-lang.zulipchat.com/#narrow/stream/266220-t-rustdoc/topic/stabilizing.20doctest.20xcompile. It sounds like there are a few minor things to decide:

The big question is if we are OK with potentially breaking projects that are doing cargo test --target …, where their doctests don't work on that target. I tried this on the standard library, and identified that arm-android failed on some tests. They look easy to fix, but are we OK with that in general? Personally I'm OK with it as long as we have some very clear notices in the release notes and blog post.