jestjs / jest

Delightful JavaScript Testing.
https://jestjs.io
MIT License
44.28k stars 6.47k forks source link

[Bug]: error in BeforeAll is propagated to all tests #15361

Open NeilRiver opened 3 weeks ago

NeilRiver commented 3 weeks ago

Version

29.7.0

Steps to reproduce

mytest

describe('Users', () => {
  let token: string;

  beforeAll(async () => {
    const auth = await api.post<Auth>('/user/auth123', user, AuthSchema);
    token = auth.data.tokens.jwt_access;
    return auth;
  });

  test('refresh token', async () => {
    const response = await api.get<RefreshType>('/user/refresh', RefreshSchema, { token });
    return response;
  });
  ....

axios config

async function makeRequest<T>({
  url,
  method,
  data = null,
  token,
  headers = {},
  params = {}
}: RequestParams): Promise<AxiosResponse<T>> {
  try {
    const finalHeaders = new AxiosHeaders({
      ...headers,
      ...(token ? { Authorization: Bearer ${token} } : {})
    });

    const response = await axiosInstance({
      url,
      method,
      data,
      headers: finalHeaders,
      params
    });

    performApiChecks(response);
    return response;
  } catch (error: any) {
    console.error('error message:', error.response ? error.response.data : error.message);
    throw parseAxiosErrors(error);
  }
}

const apiRequest = {
  get<T>(
    url: string,
    schema: z.ZodType<T>,
    options: RequestOptions = {}
  ): Promise<AxiosResponse<T>> {
    return makeRequest<unknown>({
      url,
      method: 'GET',
      ...options
    }).then((response) => ({
      ...response,
      data: validateResponseData(schema, response.data)
    }));
  },
    .....

custom reporter

onTestResult(test, testResult) {
    const completedTests =
      testResult?.testResults?.filter(
        (test) => test.duration != null && test.status !== 'passed'
      )  [];
    if (!this.results.length) return;

    const formattedTestResults = completedTests.map((test) => {
      const failureInfo =
        test.failureDetails && test.failureDetails[0] ? test.failureDetails[0] : {};

      return {
        ...failureInfo,
        openApiBlock: test.ancestorTitles && test.ancestorTitles[0] ? test.ancestorTitles[0] : '',
        testDescription: test.title  '',
        executionTime: test.duration || 0
      };
    });

    this.results.push(...formattedTestResults);
  }

Expected behavior

when running 1 test 1 test is executed

Actual behavior

all are fulfilled

Additional context

I deliberately broke the authorization before all the tests!

const auth = await api.post<Auth>('/user/authorization123', user, AuthSchema);

in order to check how the custom reporter works I pass the request data via throw into the custom reporter

throw parseAxiosErrors(error);

there is nothing interesting inside this function, I just take the fields that I need and then inside the custom reporter I try to generate json

onTestResult(test, testResult) {
    const completedTests =
      testResult?.testResults?.filter(
        (test) => test.duration != null && test.status !== 'passed'
      )  [];
    if (!this.results.length) return;

and here I see an error! it is not clear why (if I run 1 test through vscode, for example refresh token, then the error from BeforeAll will spread to all tests that I did not run!

FAIL  tests/user.test.ts (10.619 s)
   User
    ✕ refresh token (3 ms)
    ○ skipped ...
    ○ skipped ...
    ○ skipped ...
    ○ skipped ...
    ○ skipped ...
    ○ skipped ...
    ○ skipped ...
    ○ skipped ...
    ○ skipped ...
    ....
      ○ skipped ...
      ○ skipped ...
      ○ skipped ...
    ....
      ○ skipped ... 
      ○ skipped ... 
      ○ skipped ... 

  ● ..... refresh token
    Request failed with status code 404
  ● .....
    Request failed with status code 404
  ● ..... 
    Request failed with status code 404
  ● ..... 
    Request failed with status code 404
  ● ..... 
    Request failed with status code 404
  ● ..... 
    Request failed with status code 404
  ● ..... 
    Request failed with status code 404
  ● ..... 
    Request failed with status code 404
  ● ..... 
    Request failed with status code 404
  ● ..... 
    Request failed with status code 404
  ● ..... 
    Request failed with status code 404
  ● ..... 
    Request failed with status code 404
  ● ..... 
    Request failed with status code 404
  ● ..... 
    Request failed with status code 404
  ● ..... 
    Request failed with status code 404
  ● ..... 

once again... I pressed to execute only 1 test, but jest for some reason went to execute all tests at once and here in the console you see that errors started to pour in. And the worst part... in the custom reporter that I pass via throw obj: AxiosError I get its payload on all tests. Although they should be skipped!

image image Hello! What 16 tests? I launched 1

then it is not very clear why the tests are in a pending state? I called a method that is executed after the tests are completed, where does the pending come from?

and the console clearly says that the tests are skipped, but in the reporter they are pending and for some reason with a payload debugger

testResult.testResults[0].status = 'failed'
testResult.testResults[1].status = 'pending'
testResult.testResults[2].status = 'pending'
testResult.testResults[3].status = 'pending'

Can someone please tell me what's going on? cli vscode

cd /.../.../.../myproject ; /usr/bin/env 'NODE_OPTIONS= --require /usr/share/code/resources/app/extensions/ms-vscode.js-debug/src/bootloader.js  --inspect-publish-uid=http' 'VSCODE_INSPECTOR_OPTIONS=:::{"inspectorIpc":"/tmp/node-cdp.248916-f0d81ff2-51.sock","deferredMode":false,"waitForDebugger":"","execPath":"/.../.../.nvm/versions/node/v22.7.0/bin/node","onlyEntrypoint":false,"autoAttachMode":"always","fileCallback":"/tmp/node-debug-callback-e9ef16dd3f4a63df"}' /.../.../.nvm/versions/node/v22.7.0/bin/node ./node_modules/.bin/jest --runInBand --watchAll=false --testNamePattern User\ refresh\ token\$ --runTestsByPath /.../.../.../myproject/tests/user.test.ts

Environment

System:
    OS: Linux 6.11 Fedora Linux 40 (KDE Plasma)
    CPU: (12) x64 13th Gen Intel(R) Core(TM) i5-1335U
  Binaries:
    Node: 22.7.0 - ~/.nvm/versions/node/v22.7.0/bin/node
    npm: 10.8.2 - /usr/bin/npm
  npmPackages:
    jest: ^29.7.0 => 29.7.0