Tests are flakey and unstable

html-next / vertical-collection

Infinite Scroll and Occlusion at > 60FPS

MIT License

176 stars 77 forks source link

Open runspired opened 1 year ago

runspired commented 1 year ago

tests will fail in unpredictable patterns, and this is somewhat hidden by CI timing out. Obervable in #381 where a readme change leads to failures.

mixonic commented 1 year ago

There are two issues:

First, there are unpredictable failures in the test suite. These are papered over by a retry script which re-runs the whole suite on CI if there is a failure, and only gives up after several attempts.
Second, there is something in the test suite or runner which causes the github action to hang. This second item is what the timeout circumvents. Without a timeout, it simply hangs for hours.