search

oli-obk / ui_test

A test framework for testing rustc diagnostics output
24 stars 24 forks source link

Sometimes, no summary is printed at the end #238

Closed RalfJung closed 2 months ago

RalfJung commented 3 months ago

Running ./test.sh in MiniRust sometimes looks like this:

[...]
tests/pass/static.fixed ⠒ 
tests/pass/zst.fixed ⠒ 
tests/pass/const.fixed ⠒ 
tests/pass/enum_direct_tag.fixed ⠒ 
tests/pass/tuples.fixed ⠒ 
tests/pass/union.fixed ⠚ 
tests/pass/enum_niche_tag.fixed ⠚ 
tests/ub/inv_bool.fixed ⠚ 
tests/ub/unaligned.fixed ⠚ 
tests/ub/return_ref_to_local.fixed ⠚ 
tests/ub/array_out_of_bounds.fixed ⠚ 
tests/ub/nullptr.fixed ⠚ 
tests/ub/destroy_prov.fixed ⠚ 
tests/pass/is_zero.fixed ⠚ 
tests/ub/enum_mark_used_bytes.fixed ⠚     r@r-ethtop:~/src/rust/minirust (master=)$

Somehow, properly marking all tests as finished and printing the summary with the number of passed tests is skipped. This is non-deterministic, so likely a race condition.

I have not seen this happen before https://github.com/minirust/minirust/pull/188 landed.

oli-obk commented 3 months ago

Yea looks like if rendering takes too long, we may exit before finishing rendering

RalfJung commented 3 months ago

Possibly related, I am also often getting this when a test fails:

Error: tests failed

Location:
    /home/r/.cargo/registry/src/index.crates.io-6f17d22bba15001f/ui_test-0.23.0/src/lib.rs:310:13
error: test failed, to rerun pass `--test ui`

No details of the actual failure is printed. Needless to say, that makes debugging kind of tricky ;)