Jest should avoid printing the individual tests if beforeAll fails

[thernstig]

🐛 Bug Report

When beforeAll fails, none of the other tests actually execute, but they all fail with the same reason, and they all print the same error.

Jest should avoid printing the individual tests if beforeAll fails. Right now the implementation just checks if there is an error and if yes, print it and fail the test without actually executing it.

Note that @SimenB asked me to create this https://github.com/facebook/jest/issues/6695#issuecomment-619405847.

To Reproduce

Run a beforeAll() that fails where a couple of tests are executed after. This is the failure received (with the supplied repro below):

 FAIL   src/test.unit.test.js
  test that a 3rd party API remains consistent
    ✕ API function 1
    ✕ API function 2
    ✕ API function 3

  ● test that a 3rd party API remains consistent › API function 1

    expect(received).toBe(expected) // Object.is equality

    Expected: "successful"
    Received: "login"

      1 | describe('test that a 3rd party API remains consistent', () => {
    > 2 |   beforeAll(() => expect('login').toBe('successful')); // this will fail
        |                                   ^
      3 |   test('API function 1', () => expect(1).toBe(1)); // each...
      4 |   test('API function 2', () => expect(2).toBe(2)); // ...of these...
      5 |   test('API function 3', () => expect(3).toBe(3)); // ...will fail too

      at src/test.unit.test.js:2:35

  ● test that a 3rd party API remains consistent › API function 2

    expect(received).toBe(expected) // Object.is equality

    Expected: "successful"
    Received: "login"

      1 | describe('test that a 3rd party API remains consistent', () => {
    > 2 |   beforeAll(() => expect('login').toBe('successful')); // this will fail
        |                                   ^
      3 |   test('API function 1', () => expect(1).toBe(1)); // each...
      4 |   test('API function 2', () => expect(2).toBe(2)); // ...of these...
      5 |   test('API function 3', () => expect(3).toBe(3)); // ...will fail too

      at src/test.unit.test.js:2:35

  ● test that a 3rd party API remains consistent › API function 3

    expect(received).toBe(expected) // Object.is equality

    Expected: "successful"
    Received: "login"

      1 | describe('test that a 3rd party API remains consistent', () => {
    > 2 |   beforeAll(() => expect('login').toBe('successful')); // this will fail
        |                                   ^
      3 |   test('API function 1', () => expect(1).toBe(1)); // each...
      4 |   test('API function 2', () => expect(2).toBe(2)); // ...of these...
      5 |   test('API function 3', () => expect(3).toBe(3)); // ...will fail too

      at src/test.unit.test.js:2:35

Test Suites: 1 failed, 1 total
Tests:       3 failed, 3 total
Snapshots:   0 total
Time:        1.037s

Expected behavior

Jest should avoid printing the individual tests if beforeAll fails. It is highly confusing for users.

Link to repl or repo (highly encouraged)

describe('test that a 3rd party API remains consistent', () => {
  beforeAll(() => expect('login').toBe('successful')); // this will fail
  test('API function 1', () => expect(1).toBe(1)); // each...
  test('API function 2', () => expect(2).toBe(2)); // ...of these...
  test('API function 3', () => expect(3).toBe(3)); // ...will be reported as failed too
});

envinfo

  System:
    OS: Linux 4.15 Ubuntu 18.04.4 LTS (Bionic Beaver)
    CPU: (4) x64 Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
  Binaries:
    Node: 12.15.0 - ~/.config/nvm/12.15.0/bin/node
    npm: 6.13.4 - ~/.config/nvm/12.15.0/bin/npm
  npmPackages:
    jest: 25.1.0 => 25.1.0

[vldmrkl]

Hi 👋

Can I work on this issue?

[SimenB]

Yeah, that'd be lovely! Feel free to ask questions if you get stuck or want a pointer 🙂

I'd start in jest-circus (jest-jasmine is soft-deprecated, so unless you want to you can ignore it). Maybe the approach in #7201 might help you get started?

[DMeechan]

When beforeAll fails, it also ignores it.only and logs all tests as failing: https://repl.it/repls/MobileBaggyLearning

Wasn't sure if this bug will already be fixed by #10004 so please let me know if you'd like me to file a new issue!

[PizzaPartyInc]

Hi! Has there been any progress on this issue? :)

[thernstig]

@vldmrkl Looking forward to you working on this issue 😄

[vldmrkl]

Sorry, can't continue working on this issue. My latest progress could be found at #10004 I apologize for miscommunication.

[github-actions[bot]]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

[markdoliner-doma]

I think this change still makes sense and the issue should remain open.

[itaizelther]

Hey, I'd like to work on this issue. :)

[thernstig]

@itaizelther I'd say go for it and push a PR, continuing on the one already started.

[itaizelther]

@thernstig I have no write permissions to the existing PR, so I have created a new one: #13273 It is done, you may review it

[thernstig]

I am not a maintainer here, just meant that I don't think anyone minds if you start a new PR. Great that you did!

[vldmrkl]

@itaizelther thank you 🙏

[github-actions[bot]]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

[thernstig]

Since there is a PR at https://github.com/jestjs/jest/pull/13273 I am bumping this

[github-actions[bot]]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

[thernstig]

Since there is a PR at https://github.com/jestjs/jest/pull/13273 I am bumping this