Closed mtrezza closed 2 weeks ago
@mtrezza I was just looking into doing something like this. Using the test name as an identifier would be the easiest way to go.
There may be ways; I've looked into this quite extensively with jasmine but I couldn't find a way given its architecture. Hence I published https://github.com/parse-community/parse-server/pull/8655 to document my findings, closed it and kept this issue open, in case someone finds a way in the future. I think I've spent more than a day on it. The challenge is to build something on top of, or integrated into jasmine, so we have the test stats, etc. It may not be difficult to re-run a test, but I couldn't find a way to do that so that jasmine understands what's happening.
Another approach that came to my mind was to maybe use the GitHub CI to automatically create a test run per spec file. So that instead of re-running all tests when one fails, only the spec file of the failing test needs to re-run. But this'd need to consider coverage reporting, not spamming our CI dashboard, etc.
🎉 This change has been released in version 7.3.0-alpha.7
New Feature / Enhancement Checklist
Current Limitation
There are still some flaky tests;
ci: failed
labels on READMEFeature / Enhancement Description
Add a test-retry logic on jasmine-level (not GitHub workflow level) specifically for flaky tests.
Alternatives / Workarounds
Manually re-run tests (status quo)
References