framesurge / perseus

A state-driven web development framework for Rust with full support for server-side rendering and static generation.
https://framesurge.sh/perseus/en-US
MIT License
2.15k stars 89 forks source link

[Meta] Fix i18n end-to-end test #214

Closed arctic-hen7 closed 1 year ago

arctic-hen7 commented 1 year ago

For about a year now, the i18n end-to-end test has been extremely patchy, and, since GHA automatically stops other test runs if one fails, it's not uncommon for a PR or commit to get failing tests even if it's perfectly valid, just because the i18n test fails. Others have failed in the past (false negatives are an unavoidable side-effect of full headless browser end-to-end tests, I get that), but none as frequently as the i18n one (to my observation).

In terms of fixing this, I think the first step is to try to get GHA to push ahead and run all the tests, even if a few fail, which would also provide far better debugging capability, and make re-runs much speedier (we usually just re-run failing tests, and if there's only one rather than 14 others that were cancelled, that's much quicker!). Beyond that, I'll have a look in the coming days at any possible improvements that could be made to the test itself to prevent the weird failure.

TODO actual error log when I have time to extract it from GHA

github-actions[bot] commented 1 year ago

This repo is set up to triage issues with Tribble, but this issue couldn't be processed. If you deliberately didn't use Tribble to report this issue, you can safely ignore this warning. If you did, something's gone wrong here.

arctic-hen7 commented 1 year ago

Error log:

---- main stdout ----
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `""`,
 right: `"Hello, \u{2068}User\u{2069}!"`', examples/core/i18n/tests/main.rs:16:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
arctic-hen7 commented 1 year ago

Also, this is less inconvenient now that the matrix is set up to not fail-fast, meaning everything is run anyway so we can actually see exactly what failed each time.

arctic-hen7 commented 1 year ago

Noting that I haven't seen this bug since capsules were merged, so I may have fixed something at some stage accidentally...closing very tentatively for now...

arctic-hen7 commented 1 year ago

Well, that was predictable.