search

FredrikNoren / ungit

The easiest way to use git. On any platform. Anywhere.
MIT License
10.42k stars 637 forks source link

Fix intermittent test failures #1495

Closed jung-kim closed 2 years ago

jung-kim commented 2 years ago

Test fails intermittently quite often and considering various combinations of tests we run, test usually never passes.

https://github.com/FredrikNoren/ungit/runs/4005712725?check_suite_focus=true

This maybe simple as doing more retries during tests

Screen Shot 2021-10-25 at 10 50 35 PM

campersau commented 2 years ago

I know macos clicktests are very flaky with GitHub Actions. On travis (which we need to migrate to .com) they often pass like here https://travis-ci.org/github/FredrikNoren/ungit/builds/772284376. That's why I have added a retry for clicks in https://github.com/FredrikNoren/ungit/pull/1336/commits/934d152a2609f62a29223dc4cb96df7c9a4a04dd: https://github.com/FredrikNoren/ungit/blob/d612c0f2eb859c366b227dac628fe2ba45df3d7d/clicktests/environment.js#L269-L282

But since I don't have a mac these issues are very hard to debug for me.

jung-kim commented 2 years ago

I think number of tests has gotten bit out of control. I don't there is a much value in testing in so many node versions and OS versions. To me, node works on an OS, then it should be enough. This is because if a test fails on specific OS, then it's a library or node issue, not ungit issue.

If we were to have tests within different browsers, then that would be helpful. But not various tests for different OSs.

I think we should cut down on appveyor and travis and stick with github actions and make it stable

jung-kim commented 2 years ago

Closing this issues now. #1505 has more details on current state of click testing.

While there are still some outstanding race conditions, tests has stabilized quite a lot.