Jest tests failing on Github Action but pass locally

[irverysmart]

🐛 Bug Report

Context: I have about ~100 test suites with a total of ~600 tests (which contain asynchronous tests as well as synchronous tests). I'm using yarn test --coverage to run the tests and generate a coverage report, and when executing this command locally all of the tests pass (and the coverage report is created) as expected.

However, I'm attempting to generate a test coverage report in a Github Action in order to upload the coverage report to a code coverage tool for analysis. To do this, I'm running yarn test --coverage in the Github Action. When executing this command in the Github Action, it has failed 100% of the time (with 2-3 specific tests always being the cause for the failure). For reference, this is my Github Action:

name: Example Action

on:
  pull_request:
    types: [opened, reopened, synchronize]
  push:
    branches:
      - main

jobs:
  nodejs:
    name: Example Action
    runs-on: ubuntu-latest
    timeout-minutes: 20

    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Set Node Version
        uses: actions/setup-node@v2
        with:
          node-version: 12
      - name: yarn install
        run: yarn install
      - name: yarn coverage
        run: yarn test --coverage

I investigated into why this would happen and realized that it could possibly be due to the "low-resources" available on the machine running the Github Action. In other words, since the container running the Github Action has fewer cores to work with, it seems like Jest is only allowing 1 worker thread to be spawned (as opposed to the many threads that would be spawned locally to run the tests). To test if this was possibly the issue, I ran yarn test --coverage --maxWorkers=1 locally, and as I expected the same tests that were failing on the Github Action were now failing locally (however now it was only 1-2 tests failing of the original 2-3 tests, depending on the run).

My question is, why would the number of threads being used by Jest effect the passing/failing of a test? And also, does anyone have any guidance on any way to investigate/fix this issue? Also, is this expected behavior?

To Reproduce

Steps to reproduce the behavior:

1. Run jest test --coverage locally - all tests pass.
2. Run jest test --coverage in Github Actions, some tests intermittently fail.

Expected behavior

1. Run jest test --coverage locally - all tests pass.
2. Run jest test --coverage in Github Actions - all tests should pass.

[kevcenteno]

We're able to reproduce this in this branch: https://github.com/maxmind/minfraud-api-node/tree/upgrade-node-modules

Also, adding the --runInBand causes a coverage failure

[maheshsundaram]

@irverysmart I am glad I discovered this issue. I am having the same problem when running jest tests with playwright in GitHub Actions. I wasn't able to figure out why they would pass locally but when running in Actions they hang with the message Jest did not exit one second after the test run has completed.

[fitsallun]

I'm seeing the same when using setSystemTime and useFakeTimers in a test.

[kian2attari]

+1 also seeing the same issue with .toBeCalledTimes(1). Works locally, but fails on CI as the mocked function in question doesn't get called

[williscool]

Sharing what helped me debug this here

const wtf = require('wtfnode');
const whyRunNodelog = require('why-is-node-running') // should be your first require

// ... rest of your require statements and such ....

afterAll(async () => {
  wtf.dump()
  whyRunNodelog()
  console.log('process._getActiveHandles()', process._getActiveHandles())
});

// rest of your code ...

https://github.community/t/jest-tests-hang/17537/9?u=williscool

[EmmaB]

+1

[huvber]

+1

[zacdemi]

+1

[maheshsundaram]

@irverysmart I am glad I discovered this issue. I am having the same problem when running jest tests with playwright in GitHub Actions. I wasn't able to figure out why they would pass locally but when running in Actions they hang with the message Jest did not exit one second after the test run has completed.

For me the issue was related to number of workers locally vs. in CI

[SimenB]

All Jest does is use os.cpus() and run the amount of workers that correspond to the number of cores (see #10653 for some details on how we run our own tests).

Beyond that there's not much we can do.

My question is, why would the number of threads being used by Jest effect the passing/failing of a test?

Your tests might do something which only works out in a worker or vice versa. E.g. concurrent DB access or some such. Or passing of data between processes.

And also, does anyone have any guidance on any way to investigate/fix this issue? Also, is this expected behavior?

It is not expected that your tests behave differently depending on whether or not the run in band or not.

---

This issue isn't really actionable unless somebody can put together a minimal reproduction. And even then, I doubt there's something to change in Jest

[Galzk]

I've had the same issue and it turned out to be caused by differences between my local time (or time format, e.g DD/MM/YYYY vs MM/DD/YYYY) while trying to fill a input with type="time" / "date" If you're using moment.js or similar, make sure to strictly set the timezone

[SimenB]

Right, that sounds like a typical error. And not something Jest can detect and/or warm about.

So I'll close this. Doubtful any such issues are jest's "fault", but if examples are found, please open up a jew issue.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.