Jest tests hang when mocking a stream response

[jrnail23]

Prerequisites

• [X] I confirm my issue is not in the opened issues
• [X] I confirm the Frequently Asked Questions didn't contain the answer to my issue

Environment check

• [X] I'm using the latest msw version
• [X] I'm using Node.js version 18 or higher

Node.js version

v18.19.0

Reproduction repository

https://github.com/jrnail23/msw-hang-repro

Reproduction steps

npm install then npm test

Current behavior

When repro.test.js is run, Test passes, but Jest never exits. When only normal.test.js is run (npm test -- --testPathPattern=normal), test passes, and Jest exits cleanly.

 PASS  ./repro.test.js
  ✓ reproduce openHandles issue (17 ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        0.272 s, estimated 1 s
Ran all test suites.
Jest did not exit one second after the test run has completed.

'This usually means that there are asynchronous operations that weren't stopped in your tests. Consider running Jest with `--detectOpenHandles` to troubleshoot this issue.

Expected behavior

Jest should exit cleanly when test run includes repro.test.js

[jrnail23]

Note: --detectOpenHandles doesn't show any warning, and the process still hangs.

[jrnail23]

Another note: I tried switching to Axios instead of Supertest, and the problem still remains, so it's not likely an issue with Superagent/Supertest

[keitakn]

I am also experiencing a similar issue.

The version of Node.js I am using is v18.19.0.

The test code is as follows:

/**
 * @jest-environment node
 */
import { generateCatMessage } from '@/api/client/generateCatMessage';
import { TooManyRequestsError } from '@/api/errors';
import {
  isGenerateCatMessageResponse,
  type GenerateCatMessageResponse,
} from '@/features';
import { createInternalApiUrl } from '@/features/url';
import {
  mockGenerateCatMessage,
  mockGenerateCatMessageTooManyRequestsErrorResponseBody,
} from '@/mocks';
import { afterAll, beforeAll, afterEach, describe, expect, it } from '@jest/globals';
import { http } from 'msw';
import { setupServer } from 'msw/node';

const mockHandlers = [
  http.post(createInternalApiUrl('generateCatMessage'), mockGenerateCatMessage),
];

const mockServer = setupServer(...mockHandlers);

const extractResponseBody = (
  response: Response,
): ReadableStream<Uint8Array> => {
  if (response.body === null) {
    throw new Error('generatedResponse.body is null');
  }

  return response.body;
};

// eslint-disable-next-line
describe('src/api/client/generateCatMessage.ts generateCatMessage TestCases', () => {
  beforeAll(() => {
    mockServer.listen();
  });

  afterEach(() => {
    mockServer.resetHandlers();
  });

  afterAll(() => {
    mockServer.close();
  });

  it('should be able to generated CatMessage', async () => {
    const generatedResponse = await generateCatMessage({
      catId: 'moko',
      userId: 'userId1234567890',
      message: 'こんにちは！',
    });

    expect(generatedResponse.body).toBeInstanceOf(ReadableStream);

    const generatedResponseBody: ReadableStream<Uint8Array> =
      extractResponseBody(generatedResponse);

    const expected = [
      {
        conversationId: '7fe730ac-5ea9-d01d-0629-568b21f72982',
        message: 'こんにちは🐱',
      },
      {
        conversationId: '7fe730ac-5ea9-d01d-0629-568b21f72982',
        message: 'もこだにゃん🐱',
      },
      {
        conversationId: '7fe730ac-5ea9-d01d-0629-568b21f72982',
        message: 'お話しようにゃん🐱',
      },
      {
        conversationId: '7fe730ac-5ea9-d01d-0629-568b21f72982',
        message: '🐱🐱🐱',
      },
    ];

    const reader = generatedResponseBody.getReader();
    const decoder = new TextDecoder();

    let index = 0;

    const readStream = async (): Promise<undefined> => {
      const { done, value } = await reader.read();

      if (done) {
        return;
      }

      const objects = decoder
        .decode(value)
        .split('\n\n')
        .map((line) => {
          const jsonString = line.trim().split('data: ')[1];
          try {
            const parsedJson = JSON.parse(jsonString) as unknown;

            return isGenerateCatMessageResponse(parsedJson) ? parsedJson : null;
          } catch {
            return null;
          }
        })
        .filter(Boolean) as GenerateCatMessageResponse[];

      for (const object of objects) {
        expect(object).toStrictEqual(expected[index]);
        index++;
      }

      await readStream();
    };

    await readStream();

    reader.releaseLock();
  }, 10000);

  it('should TooManyRequestsError Throw, because unexpected response body', async () => {
    mockServer.use(
      http.post(
        createInternalApiUrl('generateCatMessage'),
        mockGenerateCatMessageTooManyRequestsErrorResponseBody,
      ),
    );

    const dto = {
      catId: 'moko',
      userId: 'userId1234567890',
      message: 'ねこ！',
    } as const;

    await expect(generateCatMessage(dto)).rejects.toThrow(TooManyRequestsError);
  });
});

expect(object).toStrictEqual(expected[index]); is working as expected, but Jest is not terminating normally.

The following warning message is being displayed:

A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --detectOpenHandles to find leaks. Active timers can also cause this, ensure that .unref() was called on them.

Even when I run Jest with the --detectOpenHandles option, the warning is not displayed.

Mocks are defined as follows:

import { sleep } from '@/utils';
import { HttpResponse, type ResponseResolver } from 'msw';

const encoder = new TextEncoder();

export const mockGenerateCatMessage: ResponseResolver = () => {
  const stream = new ReadableStream({
    start: async (controller) => {
      await sleep();

      controller.enqueue(
        encoder.encode(
          'data: {"conversationId": "7fe730ac-5ea9-d01d-0629-568b21f72982", "message": "こんにちは🐱"}',
        ),
      );

      await sleep(0.5);

      controller.enqueue(
        encoder.encode(
          'data: {"conversationId": "7fe730ac-5ea9-d01d-0629-568b21f72982", "message": "もこだにゃん🐱"}',
        ),
      );

      await sleep(0.5);

      controller.enqueue(
        encoder.encode(
          'data: {"conversationId": "7fe730ac-5ea9-d01d-0629-568b21f72982", "message": "お話しようにゃん🐱"}',
        ),
      );

      await sleep(0.5);

      controller.enqueue(
        encoder.encode(
          'data: {"conversationId": "7fe730ac-5ea9-d01d-0629-568b21f72982", "message": "🐱🐱🐱"}',
        ),
      );
      controller.close();
    },
  });

  return new HttpResponse(stream, {
    headers: {
      'Content-Type': 'text/event-stream',
    },
  });
};

Additional information: These test codes are being used in our project. You can check the versions of msw and jest by looking at the project's package.json.

https://github.com/nekochans/ai-cat-frontend

[kettanaito]

@keitakn, thanks for providing a reproduction repository. In your project, you import whatwg-fetch for tests, which is a no-op. That's a polyfill that you don't need if you're running Node.js v18+.

https://github.com/nekochans/ai-cat-frontend/blob/e9c42caef32e9bd129cfa2a85963009ddc7e70c6/jest.setup.ts#L3

You should remove this. All sorts of issues can occur if you are running non-standard fetch. I suspect that's the culprit behind the stream response pending forever. Here's what you should do:

1. Ensure you are using Node.js v18+ (you cannot use MSW 2.x without this anyway).
2. Remove whatwg-fetch from your tests. You no longer need this.
3. Retry the tests. Chances are, it's going to work.

If you're having issues with Jest in modern Node.js, follow the suggestions from the Migration guidelines to resolve them.

Note that there's a recent issues that makes Jest incapable of understanding ReadableStream when using the latest version of undici (#1931). Until Jest migrates from core-js polyfilling structuredClone, you cannot use ReadableStreams in Jest. This isn't a problem in modern test runners, like Vitest, so I highly encourage you use those instead.

Conclusion

I understand your frustration with things not working as you expect. MSW by itself doesn't do anything with streams. It doesn't do anything with fetch, requests, or responses. All those are standard APIs used by you and your test/development environment. It so happens that some tools are rather archaic and rely on polyfills for things that have been standard and shipping in both browser and Node.js for years. Those tools bring you down. Migrate from those tools, please.

[jrnail23]

I think you may have closed this prematurely. I'm the OP, and my reproduction has nothing to do with whatwg-fetch, or even fetch at all, for that matter.

  "dependencies": {
    "jest": "^29.7.0",
    "msw": "^2.0.12",
    "supertest": "^6.3.3"
  }

[kettanaito]

@jrnail23, you use Jest. I also think you use JSDOM, otherwise you wouldn't have had this issue. JSDOM does two things that are rather bad for you:

1. Strips away all Node.js globals, like fetch and ReadableStream;
2. Depends on core-js to polyfill standard Node.js API.

There are all sorts of things that can go wrong in this scenario. I made a decision not to support tools that don't rely on Node.js/JavaScript and instead prefer polluting your tests with polyfills. MSW doesn't own your fetch, so whichever fetch is present in your test environment is the cause of the problem.

[jrnail23]

I've tried with and without JSDOM. Same results.

[kettanaito]

@jrnail23, I highly recommend you see our Usage examples that feature MSW with Jest and Jest+JSDOM and see how they differ from your setup. Let me know what you find!

[jrnail23]

will do, thanks for the tip

[jrnail23]

@kettanaito I tried emulating what you've got in the jest examples, and the result is unchanged -- still hangs upon test run completion when the stream test runs. I've pushed up a new alternate branch to my reproduction repo.

[keitakn]

@kettanaito

I recently migrated from Jest to Vitest and it now works without any issues.🙌

https://github.com/nekochans/ai-cat-frontend/pull/78

I had been interested in Vitest for a while but was hesitant about the migration cost. Your comment was the catalyst for my decision, and I'm glad that I made the switch.

msw has been very useful for my project.

Thank you for developing msw.🙏

I plan to continue using it in my projects.

[kettanaito]

@keitakn, I'm glad to hear migrating to modern tooling helped.