e2e tests are unstable

[khassel]

No new problem, our e2e test are failing unrelated to the tested code.

Was already discussed in some PR's so I decided to make an issue for this.

My observations:

• not related to native fetch, was the same with node-fetch before
• most fails are with node v20 (but not exclusive)
• seems to be a timing problem, very hard to reproduce locally
• looks like different tests interfere, maybe the next one is started before the predecessor stopped

• some strange errors are mocked in tests/e2e/helpers/mock-console.js:

  const mockError = (err) => {
  if (
    err.includes("ECONNREFUSED") ||
    err.includes("ECONNRESET") ||
    err.includes("socket hang up") ||
    err.includes("exports is not defined") ||
    err.includes("write EPIPE") ||
    err.includes("AggregateError") ||
    err.includes("ERR_SOCKET_CONNECTION_TIMEOUT")
  ) {
    jest.fn();
  } else {
    console.dir(err);
  }
  };
  
  global.console = {
  log: jest.fn(),
  dir: console.dir,
  error: mockError,
  warn: console.warn,
  info: jest.fn(),
  debug: console.debug
  };

  So removing mockError here will show these errors and may can lead to the real error

[bugsounet]

most fails are with node v20 (but not exclusive)

Maybe it's related to node 20 version, this branch is still young

Just for see: Test suite Passed / Failed node v20.x version:

[Pass] node v20.7.0

[Failed] node v20.6.1

[khassel]

yes, since running on v20.7.0 it is more stable.

[bugsounet]

this is not the case with v20.8.0

[bugsounet]

Finally, I went back to node v18. Too many strange problems on my Pi4. but oddly less on my dev PC