ReferenceError: TextEncoder is not defined (updating to v2)

Danielvandervelden commented 10 months ago

Prerequisites

[X] I confirm my issue is not in the opened issues
[X] I confirm the Frequently Asked Questions didn't contain the answer to my issue

Environment check

[X] I'm using the latest msw version
[X] I'm using Node.js version 18 or higher

Node.js version

18.16.0

Reproduction repository

https://github.com/Danielvandervelden/msw-issue

Reproduction steps

npm i && npm run test

Current behavior

src/example.test.ts
  ● Test suite failed to run

    ReferenceError: TextEncoder is not defined

      1 | import "@testing-library/jest-dom";
    > 2 | import { HttpResponse, http } from "msw";
        | ^
      3 | import { setupServer } from "msw/node";
      4 |
      5 | const server = setupServer();

      at Object.<anonymous> (node_modules/@mswjs/interceptors/lib/node/chunk-3LFH2WCF.js:2:15)
      at Object.<anonymous> (node_modules/@mswjs/interceptors/lib/node/index.js:7:24)
      at Object.<anonymous> (node_modules/msw/lib/core/utils/matching/matchRequestUrl.js:26:27)
      at Object.<anonymous> (node_modules/msw/lib/core/handlers/HttpHandler.js:51:30)
      at Object.<anonymous> (node_modules/msw/lib/core/http.js:24:26)
      at Object.<anonymous> (node_modules/msw/lib/core/index.js:38:19)
      at Object.<anonymous> (src/example.test.ts:2:1)

Expected behavior

No error, the tests should pass, we are only registering a route to the server in an individual it function.

oschwede commented 10 months ago

In the migration guide there is a section about node globals not being available in jsdom environments. You can try using the suggested solution there to also make TextEncoder and TextDecoder available.

https://mswjs.io/docs/migrations/1.x-to-2.x#remap-fetch-api-globals

naghtigal commented 10 months ago

In the migration guide there is a section about node globals not being available in jsdom environments. You can try using the suggested solution there to also make TextEncoder and TextDecoder available.

https://mswjs.io/docs/migrations/1.x-to-2.x#remap-fetch-api-globals

I just was trying this as was suggested, but it doesn't really help:

In my jest.config.js: globals: { TextEncoder: TextEncoder, TextDecoder: TextDecoder },

still produce ReferenceError: TextEncoder is not defined

joel-daros commented 10 months ago

I have the same issue with a Create React App:

Reproduction Repository:

https://github.com/joel-daros/msw2-text-encoder-issue

 FAIL  src/App.test.tsx
  ● Test suite failed to run

    ReferenceError: TextEncoder is not defined

    > 1 | import { setupServer } from "msw/node";
        | ^

Danielvandervelden commented 10 months ago

In the migration guide there is a section about node globals not being available in jsdom environments. You can try using the suggested solution there to also make TextEncoder and TextDecoder available.

https://mswjs.io/docs/migrations/1.x-to-2.x#remap-fetch-api-globals

Thank you for your reply. I have indeed seen this, but unfortunately this does not solve the issue. I have updated the repository to include the globals to showcase this.

From my experimentation I'm noticing that when I explicitly remove the testEnvironment: "jest-environment-jsdom" from the jest.config.js it starts working. You can tests this in the repo as well. However, for my unit tests I need this.

Perhaps there's something happening within the jest-environment-jsdom package that is overriding something within msw, however I do not know what.

mbaumanndev commented 10 months ago

As a workaround, I managed to make it work by adding this to my jest setupTest:

import { TextEncoder } from 'node:util'

global.TextEncoder = TextEncoder

kettanaito commented 10 months ago

Solution

I've updated the docs to include the fix:

Request/Response/TextEncoder is not defined (Jest)

Please follow these instructions to have Jest set up correctly.

naghtigal commented 10 months ago

@kettanaito , thanks, it works.

Side note, there is a mistype in doc

Reflect.set(globalThis, 'Respose', Respose)

should be

Reflect.set(globalThis, 'Response', Response)

grzegorz-zadora commented 10 months ago

@Danielvandervelden

Thank you for your reply. I have indeed seen this, but unfortunately this does not solve the issue. I have updated the repository to include the globals to showcase this.

From my experimentation I'm noticing that when I explicitly remove the testEnvironment: "jest-environment-jsdom" from the jest.config.js it starts working. You can tests this in the repo as well. However, for my unit tests I need this.

Perhaps there's something happening within the jest-environment-jsdom package that is overriding something within msw, however I do not know what.

Maybe you are using the import statement instead of the require() function in the polyfills file?

The modules in the polyfills file must be loaded in the correct order, so require() is the only option.

sernaferna commented 10 months ago

Solution

I've updated the docs to include the fix:

Request/Response/TextEncoder is not defined (Jest)

Please follow these instructions to have Jest set up correctly.

This solution doesn't work with create-react-app applications, where you can't specify setupFiles for Jest. The jest.config.js file is ignored, and setting the setupFiles param in package.json spits out the error below. Is there a way to get msw working that doesn't require ejecting a CRA app?

Out of the box, Create React App only supports overriding these Jest options:

  • clearMocks
  • collectCoverageFrom
  • coveragePathIgnorePatterns
  • coverageReporters
  • coverageThreshold
  • displayName
  • extraGlobals
  • globalSetup
  • globalTeardown
  • moduleNameMapper
  • resetMocks
  • resetModules
  • restoreMocks
  • snapshotSerializers
  • testMatch
  • transform
  • transformIgnorePatterns
  • watchPathIgnorePatterns.

These options in your package.json Jest configuration are not currently supported by Create React App:

  • setupFiles

If you wish to override other Jest options, you need to eject from the default setup. You can do so by running npm run eject but remember that this is a one-way operation. You may also file an issue with Create React App to discuss supporting more options out of the box.

grzegorz-zadora commented 10 months ago

@sernaferna it appears that using craco is the sole way to avoid ejecting in your case

mattcosta7 commented 10 months ago

Solution

I've updated the docs to include the fix:

Request/Response/TextEncoder is not defined (Jest)

Please follow these instructions to have Jest set up correctly.

This solution doesn't work with create-react-app applications, where you can't specify setupFiles for Jest. The jest.config.js file is ignored, and setting the setupFiles param in package.json spits out the error below. Is there a way to get msw working that doesn't require ejecting a CRA app?
Out of the box, Create React App only supports overriding these Jest options:

  • clearMocks
  • collectCoverageFrom
  • coveragePathIgnorePatterns
  • coverageReporters
  • coverageThreshold
  • displayName
  • extraGlobals
  • globalSetup
  • globalTeardown
  • moduleNameMapper
  • resetMocks
  • resetModules
  • restoreMocks
  • snapshotSerializers
  • testMatch
  • transform
  • transformIgnorePatterns
  • watchPathIgnorePatterns.

These options in your package.json Jest configuration are not currently supported by Create React App:

  • setupFiles

If you wish to override other Jest options, you need to eject from the default setup. You can do so by running npm run eject but remember that this is a one-way operation. You may also file an issue with Create React App to discuss supporting more options out of the box.

You can always avoid react-scripts test as the test command, and instead use jest directly there, with a custom jest.config.js file. You may need to copy some of the create-react-app jest config, but that's likely pretty straightforward.

patch-package is probably the simplest option, or something like craco.

Migrating off create-react-app to a non-dead project, and one that allows for proper configuration of tools would be ideal.

Create React App often recommended forking to customize in the past, and that's also viable

joel-daros commented 10 months ago

@sernaferna it appears that using craco is the sole way to avoid ejecting in your case

Unfortunately craco doesn’t work either. I’ve updated my reproduction repo with Craco, and even after adding a custom craco config for Jest, the test still get stuck when useFakeTimers() is used .

https://github.com/joel-daros/msw2-text-encoder-issue

// craco.config.ts

import { CracoConfig } from "@craco/types";

const cracoConfig: CracoConfig = {
  jest: {
    configure: (jestConfig, { env, paths, resolve, rootDir }) => {
      return {
        ...jestConfig,
        setupFiles: [`${rootDir}/src/jest.polyfills`],
      };
    },
  },
};

export { cracoConfig as default };

I know that CRA is dead and outdated package, but it's still widely used in many projects. I don’t have the option to eject our apps, because there are some many other internal factors involved.

We might think in a different solution, otherwise, I think this can be a big barrier to migrate to MSW 2 or even the main reason to move to other alternatives in the long run.

sernaferna commented 10 months ago

Well it was a journey, but I got there in the end.

Using craco allowed me to use the setupFiles option, and get me past this issue per the msw documentation on setting up polyfills
... which led to an unexpected token: export error, which I fixed from advice I got on this issue: #1810
... which led to relative URLs not being resolved, which I fixed with advice I got from this issue: #1625
... which led to everything working, except my mocked APIs were never being called. But I realised I'd been working on this update for so long that there had been three patch releases since I started, so getting up to 2.0.4 fixed it.

RobinHerbots commented 9 months ago

@sernaferna it appears that using craco is the sole way to avoid ejecting in your case

Unfortunately craco doesn’t work either. I’ve updated my reproduction repo with Craco, and even after adding a custom craco config for Jest, the test still get stuck when useFakeTimers() is used .

https://github.com/joel-daros/msw2-text-encoder-issue

// craco.config.ts
import { CracoConfig } from "@craco/types";

const cracoConfig: CracoConfig = {
  jest: {
    configure: (jestConfig, { env, paths, resolve, rootDir }) => {
      return {
        ...jestConfig,
        setupFiles: [`${rootDir}/src/jest.polyfills`],
      };
    },
  },
};

export { cracoConfig as default };
I know that CRA is dead and outdated package, but it's still widely used in many projects. I don’t have the option to eject our apps, because there are some many other internal factors involved.

We might think in a different solution, otherwise, I think this can be a big barrier to migrate to MSW 2 or even the main reason to move to other alternatives in the long run.

You can solve it by putting the polyfills in different files.

import { TextDecoder, TextEncoder } from 'node:util';
Reflect.set(globalThis, 'TextDecoder', TextDecoder);

import { Blob, File } from 'node:buffer';

Reflect.set(globalThis, 'Blob', Blob);
Reflect.set(globalThis, 'File', File);

and so on ... and in your craco.config you add:


   jestConfig.setupFiles = [
        `${rootDir}/src/jest.polyfills1.js`,
        `${rootDir}/src/jest.polyfills2.js`,
        `${rootDir}/src/jest.polyfills3.js`
      ];```

sernaferna commented 9 months ago

I didn't need to create multiple polyfill files, it worked with just one for me, but the other thing I'd mention is that there have been a few releases lately that broke my tests, causing me to spend a few hours troubleshooting, only to get the next build and it would start working again. So if you've followed all of the advice in the comments above and it's still not working, try

Getting the latest version, or
If you already have the latest version, reverting back one patch release

gopal-panigrahi commented 9 months ago

Solution

I've updated the docs to include the fix:

Request/Response/TextEncoder is not defined (Jest)

Please follow these instructions to have Jest set up correctly.

This solution doesn't work with create-react-app applications, where you can't specify setupFiles for Jest. The jest.config.js file is ignored, and setting the setupFiles param in package.json spits out the error below. Is there a way to get msw working that doesn't require ejecting a CRA app?
Out of the box, Create React App only supports overriding these Jest options:

  • clearMocks
  • collectCoverageFrom
  • coveragePathIgnorePatterns
  • coverageReporters
  • coverageThreshold
  • displayName
  • extraGlobals
  • globalSetup
  • globalTeardown
  • moduleNameMapper
  • resetMocks
  • resetModules
  • restoreMocks
  • snapshotSerializers
  • testMatch
  • transform
  • transformIgnorePatterns
  • watchPathIgnorePatterns.

These options in your package.json Jest configuration are not currently supported by Create React App:

  • setupFiles

If you wish to override other Jest options, you need to eject from the default setup. You can do so by running npm run eject but remember that this is a one-way operation. You may also file an issue with Create React App to discuss supporting more options out of the box.

For create-react-app, without using craco or without ejecting the project, I solved it by following the below steps :

Created a jest.polyfills.js file as specified in Request/Response/TextEncoder is not defined (Jest)

Added following import statement in the default setupTests.js file in create-react-app project.


// jest-dom adds custom jest matchers for asserting on DOM nodes.
// allows you to do things like:
// expect(element).toHaveTextContent(/react/i)
// learn more: https://github.com/testing-library/jest-dom
import "@testing-library/jest-dom";
import "./test/jest.polyfills";
import { server } from "./test/server";

beforeAll(() => server.listen({ onUnhandledRequest: "error" })); beforeEach(() => server.resetHandlers()); afterAll(() => server.close());

3. Added transformIgnorePatterns in jest option in package.json

"jest": { "transformIgnorePatterns": [ "/node_modules/(?!(@bundled-es-modules)/).*/" ] }


4. And *voila*, the tests starts to intercept the requests.

> **Note: Same can be followed for typescript**

joel-daros commented 9 months ago

Solution

I've updated the docs to include the fix:

Request/Response/TextEncoder is not defined (Jest)

Please follow these instructions to have Jest set up correctly.

I don't think this works anymore in Undici 6.x

All my test failing with the error:

ReferenceError: ReadableStream is not defined

marliesparzinski commented 9 months ago

Solution

I've updated the docs to include the fix:

Request/Response/TextEncoder is not defined (Jest)

Please follow these instructions to have Jest set up correctly.

I don't think this works anymore in Undici 6.x

All my test failing with the error:
ReferenceError: ReadableStream is not defined

I found this, it might help for you as well: https://github.com/langchain-ai/langchainjs/issues/2815

So basically I added this to the jest.polyfills.js file:

import { ReadableStream } from 'node:stream/web'
if (globalThis.ReadableStream === undefined) {
  globalThis.ReadableStream = ReadableStream
}

ddcech commented 9 months ago

@marliesparzinski I added your code to my jest.polyfills.js file. It was important to add it to the top of the file. However another error shows

DataCloneError: Transferring of ReadableStream cannot be properly polyfilled in this engine

tylerpashigian commented 9 months ago

Solution

I've updated the docs to include the fix:

Request/Response/TextEncoder is not defined (Jest)

Please follow these instructions to have Jest set up correctly.

I don't think this works anymore in Undici 6.x

All my test failing with the error:
ReferenceError: ReadableStream is not defined

Did you figure this out? Also running into this issue.

joel-daros commented 9 months ago

Yes, check the solution here: https://github.com/mswjs/msw/issues/1916#issuecomment-1850757977

Borisjavier1 commented 9 months ago

As a workaround, I managed to make it work by adding this to my jest setupTest:
import { TextEncoder } from 'node:util'

global.TextEncoder = TextEncoder

This worked for me! Project with create-react-app

yaswanthmaddula-sureify commented 8 months ago

As a workaround, I managed to make it work by adding this to my jest setupTest:
import { TextEncoder } from 'node:util'

global.TextEncoder = TextEncoder

This did not work for me. Still gives the same error. I am using react-scripts: 5.0.1

m-nathani commented 8 months ago

why is this closed ? 🤔 i am facing an error.. still facing this issue afte the official docs mentioned too.

Danielvandervelden commented 8 months ago

We as a company/team decided to just stay on v1.0. It works out of the box without having to jump through any hoops. Since we just use it for testing we won't have any severe vulnerability issues anyway. Probably we'll try to update at a later point. And that's honestly my advice for anyone else trying to upgrade.

jhaeberli commented 8 months ago

Another one here... We are using CRA with Jest. It's annoying to find out there's no solution yet. So many problems. Applied Jest polyfill, with the readableStream patch and got the unexpected token error on a DatePicker, hence, lots of tests are failing. How can I convince the management to migrate to Vitest simply because of this? I already convince them to migrate to MSW v2.0 after so little support with GraphQL. I even registered in Discord only to find out nothing useful. A big warning should be included in the migration guide for users with CRA and Jest, stating that even the published workaround is not solving the issue.

christopher-theagen commented 5 months ago

.jest/setup.js

import { TextEncoder, TextDecoder } from 'util';

global.TextEncoder = TextEncoder;
global.TextDecoder = TextDecoder;

narekboshyan commented 5 months ago

please just downgrade your msw to 1.3.3. i had similliar issues i just downgraded it to 1.3.3 and it worked as expected

dnyn92 commented 4 months ago

Solution

I've updated the docs to include the fix:

Request/Response/TextEncoder is not defined (Jest)

Please follow these instructions to have Jest set up correctly.

I have followed this guide and additionally added this to get rid of the readablestream error

import { ReadableStream } from 'node:stream/web'
if (globalThis.ReadableStream === undefined) {
  globalThis.ReadableStream = ReadableStream
}

Now I get this error: ReferenceError: clearImmediate is not defined

I cannot find a remedy. Will migrate back to 1.3.2 for now

MakotoE commented 2 months ago

I was able to fix it with these steps:

Define all node globals in the setup. Install undici. Make ReadableStream, TransformStream, and clearImmediate global.

const {TextDecoder, TextEncoder} = require('node:util');
const {ReadableStream, TransformStream} = require('node:stream/web');
const {performance} = require('node:perf_hooks');

Object.defineProperties(globalThis, {
    TextDecoder: {value: TextDecoder},
    TextEncoder: {value: TextEncoder},
    ReadableStream: {value: ReadableStream},
    TransformStream: {value: TransformStream},
    performance: {value: performance},
});

const {Blob, File} = require('node:buffer');
const {fetch, Headers, FormData, Request, Response} = require('undici');
const {clearImmediate} = require('node:timers');

Object.defineProperties(globalThis, {
    fetch: {value: fetch, writable: true},
    Blob: {value: Blob},
    File: {value: File},
    Headers: {value: Headers},
    FormData: {value: FormData},
    Request: {value: Request},
    Response: {value: Response},
    clearImmediate: {value: clearImmediate},
});

Fix Cannot find module ‘msw/node’ with customExportConditions.

The JS ecosystem is such a joke. Dependencies are holding each other up with scotch tape. Break one and the entire tower falls. It's so bad that frameworks are recommending a new test framework to fix all issues. But when that test framework goes unsupported and meanwhile node v30.x comes out, deprecating many dependencies used in that framework.

mswjs / msw