Open DeeDeeG opened 3 years ago
I can confirm that I get the same two test failures (and only those two test failures) running this repo's tests on my local machine with atom --test ./test/
. So it's not just a CI thing.
Tested with Atom 1.57.0 stable on Ubuntu 20.04.
atom --version
Edit: If I try to run the tests concurrently with some light web browsing, I can also reproduce the flaky test mentioned above, and another related failure Git commands for CompositeGitStrategy made of [GitShellOutStrategy] --> ssh authentication --> fails the command on authentication failure:
--> Error: Timeout of 5000ms exceeded. [. . .]
These test failures all seem to revolve around credential handling/authentication problems on Linux... at least in CI/tests for this package.
(It's not immediately clear to me if this is a real-world problem or not, but I haven't tried to exercise these features IRL on Linux to see if they work or not.)
If anyone can snoop out the cause of the test failures, that would be great. Or methodically test these theoretical/CI/"test" cases IRL, with the actual github
package in actual Atom on Linux, to see if this is an IRL bug or just a bug in the tests.
Problem
A couple of this repo's tests fail on Linux at the moment. The test failures look like this in CI:
And this one is flaky as well, not sure if related:
Evidence narrowing it down to Electron 8+
I made some Atom builds and re-ran this repo's CI against those, essentially bisecting the Electron releases between Electron 6 (the last version this repo's CI passed on) and Electron 9 (the latest version used in Atom).
Results:
master
branch on Linux :x:; This is running against the official builds of Atom 1.59 Nightly and 1.58 Beta (both of which are built with Electron 9) (CI link).master
, built against Electron 8 (CI link).master
using Electron 7 (CI link)_Conclusion: Something must have changed between Electron 7 and 8 that affects a few of this package's tests on Linux.
(Note 1: The last passing CI run at this repo was just before the Electron 9 PR landed at Atom: https://github.com/atom/github/actions/runs/611232541 ran just before https://github.com/atom/atom/pull/21777 was merged. Note 2: This issue on Linux wouldn't be caught at the main Atom repo, because package tests only run on macOS at that repo.)