search

llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
28.06k stars 11.59k forks source link

[llvm-lit] Show test output when combining -v or -a with -q #106643

Open hnrklssn opened 3 weeks ago

hnrklssn commented 3 weeks ago

Lit has various flags to control output, that interact in non-obvious ways when combined: --quiet --succinct --verbose --show-all. Their order doesn't matter, they are completely separate flags.

At the moment, --quiet completely overrides --verbose and --show-all, without any sort of warning. I think it would be more useful if they could be combined such that --quiet still silences lit's note and warning diagnostics, but --verbose and --show-all still can show the output of test cases.

jh7370 commented 3 weeks ago

I'm thinking that we might want a more refined "last one wins" style approach, because I think it makes as little sense for -a/-v to produce test output when -q is enabled as the current behaviour. However, regardless, I think this probably needs an RFC for better visibility: the change has the potential to break people's existing scripts etc.

hnrklssn commented 3 weeks ago

To be clear, I specifically want to be able to combine the test output from -a/-v with the diagnostic silencing of -q, not for -a/-v to simply override -q. But fair enough on the RFC part.

RoboTux commented 2 weeks ago

I would agree with James, lit's doc has the following on --quiet:

Suppress any output except for test failures.

Therefore I would expect --quiet, --succinct, --verbose and --show-all to be mutually exclusive. Last one wins would be compatible with that. I think a separate --mask-warnings (YMMV for naming) which is implied to by --quiet would be better IMO. Where is the RFC by the way? I couldn't find it when searching discourse.

hnrklssn commented 2 weeks ago

Where is the RFC by the way? I couldn't find it when searching discourse.

I have not had time to put one together. Taking the RFC route, we might as well think over the overall output flag system. I'm thinking that it would be less confusing if they were truly mutually exclusive, which would be ensured if we introduced a --output-format= flag with a number of different options. Then --quiet, --succinct, --verbose and --show-all could be remapped to aliases for --output-format=quiet etc, and only 1 could take effect at a time.

That said I've got a bit on my plate now (I made this patch during a hackathon), so I don't know when I'll have the bandwidth to both write up a well thought through RFC and refactor this. If you're keen to see this a bit sooner I'm happy to pass the baton

RoboTux commented 2 weeks ago

I agree with --output-format and remapping existing options to it. And sorry I don't have the bandwidth either right now.