search

rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
97.67k stars 12.63k forks source link

merged doctests: regressions due to examining process arguments #130796

Open ehuss opened 2 weeks ago

ehuss commented 2 weeks ago

The crater run for merged doctests (#130285) detected a large number of regressions due to tests examining the process arguments. (I don't have an exact number, but let's say ~50 projects.)

Previously, there were no arguments, but now it gets arguments like *doctest-inner-test 0 which the user's CLI parsing code isn't expecting.

For example, using something like StructOpt::from_args will now fail.

It seems to be fairly common to have objects whose default constructor will parse the arguments from the command line.

I'm not sure if or how we should resolve that. One idea is to use a different mechanism for passing in the test information (like via an environment variable).

Meta

rustc 1.83.0-nightly (da889684c 2024-09-20)
binary: rustc
commit-hash: da889684c80508036ff036db8c159ffdcf27648a
commit-date: 2024-09-20
host: aarch64-apple-darwin
release: 1.83.0-nightly
LLVM version: 19.1.0
GuillaumeGomez commented 1 week ago

I vaguely remember that using environment variables was discussed but can't find anything except mentioning it here. Maybe it'd be a better approach indeed. Let me send a PR so we can make a crater run on it.