Closed arctic-hen7 closed 1 year ago
This repo is set up to triage issues with Tribble, but this issue couldn't be processed. If you deliberately didn't use Tribble to report this issue, you can safely ignore this warning. If you did, something's gone wrong here.
Error log:
---- main stdout ----
thread 'main' panicked at 'assertion failed: `(left == right)`
left: `""`,
right: `"Hello, \u{2068}User\u{2069}!"`', examples/core/i18n/tests/main.rs:16:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Also, this is less inconvenient now that the matrix is set up to not fail-fast, meaning everything is run anyway so we can actually see exactly what failed each time.
Noting that I haven't seen this bug since capsules were merged, so I may have fixed something at some stage accidentally...closing very tentatively for now...
Well, that was predictable.
For about a year now, the i18n end-to-end test has been extremely patchy, and, since GHA automatically stops other test runs if one fails, it's not uncommon for a PR or commit to get failing tests even if it's perfectly valid, just because the i18n test fails. Others have failed in the past (false negatives are an unavoidable side-effect of full headless browser end-to-end tests, I get that), but none as frequently as the i18n one (to my observation).
In terms of fixing this, I think the first step is to try to get GHA to push ahead and run all the tests, even if a few fail, which would also provide far better debugging capability, and make re-runs much speedier (we usually just re-run failing tests, and if there's only one rather than 14 others that were cancelled, that's much quicker!). Beyond that, I'll have a look in the coming days at any possible improvements that could be made to the test itself to prevent the weird failure.
TODO actual error log when I have time to extract it from GHA