mswjs / msw

Seamless REST/GraphQL API mocking library for browser and Node.js.
https://mswjs.io
MIT License
15.65k stars 504 forks source link

"JSON Parse error" in React Native in MSW 2 #1775

Closed sairus2k closed 7 months ago

sairus2k commented 11 months ago

Prerequisites

Environment check

Node.js version

v18.17.1

Reproduction repository

https://github.com/sairus2k/msw-reproduction

Reproduction steps

  1. Clone the repo
  2. Install dependencies with yarn
  3. Start metro bundler with yarn start
  4. Run the app with yarn ios or yarn android
  5. The error will be thrown when making a network request mocked by msw

Current behavior

The error is thrown:

SyntaxError: JSON Parse error: Unexpected end of input
Debug ``` LOG 10:13:07 am:297 [xhr] constructing the interceptor... LOG 10:13:07 am:299 [setup-server] constructing the interceptor... LOG 10:13:07 am:300 [xhr:on] adding "request" event listener: LOG 10:13:07 am:302 [async-event-emitter:on] adding "request" listener... LOG 10:13:07 am:303 [async-event-emitter:emit] emitting "newListener" event... LOG 10:13:07 am:303 [xhr:on] adding "response" event listener: LOG 10:13:07 am:304 [async-event-emitter:on] adding "response" listener... LOG 10:13:07 am:305 [async-event-emitter:emit] emitting "newListener" event... LOG 10:13:07 am:306 [setup-server:apply] applying the interceptor... LOG 10:13:07 am:307 [async-event-emitter:activate] set state to: ACTIVE LOG 10:13:07 am:307 [setup-server:apply] activated the emiter! ACTIVE LOG 10:13:07 am:308 [setup-server] retrieved global instance: undefined LOG 10:13:07 am:309 [setup-server:apply] no running instance found, setting up a new instance... LOG 10:13:07 am:309 [setup-server:setup] applying all 1 interceptors... LOG 10:13:07 am:310 [setup-server:setup] applying "_XMLHttpRequestInterceptor" interceptor... LOG 10:13:07 am:311 [xhr:apply] applying the interceptor... LOG 10:13:07 am:313 [async-event-emitter:activate] set state to: ACTIVE LOG 10:13:07 am:313 [xhr:apply] activated the emiter! ACTIVE LOG 10:13:07 am:314 [xhr] retrieved global instance: undefined LOG 10:13:07 am:315 [xhr:apply] no running instance found, setting up a new instance... LOG 10:13:07 am:316 [xhr:setup] patching "XMLHttpRequest" module... LOG 10:13:07 am:316 [xhr:setup] native "XMLHttpRequest" module patched! XMLHttpRequest LOG 10:13:07 am:317 [xhr] set global instance! xhr LOG 10:13:07 am:318 [setup-server:setup] adding interceptor dispose subscription LOG 10:13:07 am:318 [setup-server] set global instance! setup-server LOG Running "AwesomeProject" with {"rootTag":511,"initialProps":{}} LOG 10:13:07 am:441 [xhr] constructed new XMLHttpRequest LOG 10:13:07 am:443 [xhr] registered event "timeout" LOG 10:13:07 am:444 [xhr] addEventListener timeout LOG 10:13:07 am:453 [xhr:GET https://reactnative.dev/movies.json] open GET https://reactnative.dev/movies.json LOG 10:13:07 am:454 [xhr:GET https://reactnative.dev/movies.json] registered event "load" LOG 10:13:07 am:455 [xhr:GET https://reactnative.dev/movies.json] addEventListener load LOG 10:13:07 am:456 [xhr:GET https://reactnative.dev/movies.json] converting request to a Fetch API Request... LOG 10:13:07 am:456 [xhr:GET https://reactnative.dev/movies.json] converted request to a Fetch API Request! {"url":"https://reactnative.dev/movies.json","credentials":"include","headers":{"map":{}},"method":"GET","mode":null,"signal":{},"referrer":null,"bodyUsed":false,"_bodyInit":null,"_noBody":true,"_bodyText":""} LOG 10:13:07 am:458 [xhr:GET https://reactnative.dev/movies.json] emitting the "request" event for 1 listener(s)... LOG 10:13:07 am:458 [async-event-emitter:emit] emitting "request" event... LOG 10:13:07 am:459 [async-event-emitter:openListenerQueue] opening "request" listeners queue... LOG 10:13:07 am:460 [async-event-emitter:openListenerQueue] no queue found, creating one... LOG 10:13:07 am:461 [async-event-emitter:emit] appending a one-time cleanup "request" listener... LOG 10:13:07 am:461 [async-event-emitter:emit] emitting "newListener" event... LOG 10:13:07 am:462 [async-event-emitter:openListenerQueue] opening "request" listeners queue... LOG 10:13:07 am:463 [async-event-emitter:openListenerQueue] returning an exising queue: [] LOG 10:13:07 am:464 [async-event-emitter:on] awaiting the "request" listener... LOG 10:13:07 am:474 [async-event-emitter:emit] emitting "removeListener" event... LOG 10:13:07 am:475 [xhr:GET https://reactnative.dev/movies.json] awaiting mocked response... LOG 10:13:07 am:490 [async-event-emitter:on] "request" listener has resolved! LOG 10:13:07 am:491 [xhr:GET https://reactnative.dev/movies.json] all "request" listeners settled! LOG 10:13:07 am:492 [xhr:GET https://reactnative.dev/movies.json] event.respondWith called with: {"type":"default","status":200,"ok":true,"statusText":"OK","headers":{"map":{"content-type":"application/json"}},"url":"","bodyUsed":false,"_bodyInit":"{\"title\":\"The Basics - Networking\",\"description\":\"Your app fetched this from a remote endpoint!\",\"movies\":[{\"id\":\"1\",\"title\":\"Star Wars\",\"releaseYear\":\"1977\"},{\"id\":\"2\",\"title\":\"Back to the Future\",\"releaseYear\":\"1985\"},{\"id\":\"3\",\"title\":\"The Matrix\",\"releaseYear\":\"1999\"},{\"id\":\"4\",\"title\":\"Inception\",\"releaseYear\":\"2010\"},{\"id\":\"5\",\"title\":\"Interstellar\",\"releaseYear\":\"2014\"}]}","_bodyText":"{\"title\":\"The Basics - Networking\",\"description\":\"Your app fetched this from a remote endpoint!\",\"movies\":[{\"id\":\"1\",\"title\":\"Star Wars\",\"releaseYear\":\"1977\"},{\"id\":\"2\",\"title\":\"Back to the Future\",\"releaseYear\":\"1985\"},{\"id\":\"3\",\"title\":\"The Matrix\",\"releaseYear\":\"1999\"},{\"id\":\"4\",\"title\":\"Inception\",\"releaseYear\":\"2010\"},{\"id\":\"5\",\"title\":\"Interstellar\",\"releaseYear\":\"2014\"}]}"} LOG 10:13:07 am:492 [xhr:GET https://reactnative.dev/movies.json] received mocked response: 200 OK LOG 10:13:07 am:493 [xhr:GET https://reactnative.dev/movies.json] responding with a mocked response: 200 OK LOG 10:13:07 am:494 [xhr:GET https://reactnative.dev/movies.json] calculated response body length undefined LOG 10:13:07 am:495 [xhr:GET https://reactnative.dev/movies.json] trigger "loadstart" {"loaded":0} LOG 10:13:07 am:496 [xhr:GET https://reactnative.dev/movies.json] setReadyState: 1 -> 2 LOG 10:13:07 am:497 [xhr:GET https://reactnative.dev/movies.json] set readyState to: 2 LOG 10:13:07 am:498 [xhr:GET https://reactnative.dev/movies.json] triggerring "readystatechange" event... LOG 10:13:07 am:499 [xhr:GET https://reactnative.dev/movies.json] trigger "readystatechange" LOG 10:13:07 am:499 [xhr:GET https://reactnative.dev/movies.json] found a direct "readystatechange" callback, calling... LOG 10:13:07 am:500 [xhr:GET https://reactnative.dev/movies.json] setReadyState: 2 -> 3 LOG 10:13:07 am:501 [xhr:GET https://reactnative.dev/movies.json] set readyState to: 3 LOG 10:13:07 am:502 [xhr:GET https://reactnative.dev/movies.json] triggerring "readystatechange" event... LOG 10:13:07 am:502 [xhr:GET https://reactnative.dev/movies.json] trigger "readystatechange" LOG 10:13:07 am:503 [xhr:GET https://reactnative.dev/movies.json] found a direct "readystatechange" callback, calling... LOG 10:13:07 am:504 [xhr:GET https://reactnative.dev/movies.json] finalizing the mocked response... LOG 10:13:07 am:504 [xhr:GET https://reactnative.dev/movies.json] setReadyState: 3 -> 4 LOG 10:13:07 am:505 [xhr:GET https://reactnative.dev/movies.json] set readyState to: 4 LOG 10:13:07 am:506 [xhr:GET https://reactnative.dev/movies.json] triggerring "readystatechange" event... LOG 10:13:07 am:507 [xhr:GET https://reactnative.dev/movies.json] trigger "readystatechange" LOG 10:13:07 am:507 [xhr:GET https://reactnative.dev/movies.json] found a direct "readystatechange" callback, calling... LOG 10:13:07 am:508 [xhr:GET https://reactnative.dev/movies.json] trigger "load" {"loaded":0} LOG 10:13:07 am:509 [xhr:GET https://reactnative.dev/movies.json] found a direct "load" callback, calling... LOG 10:13:07 am:510 [xhr:GET https://reactnative.dev/movies.json] getAllResponseHeaders LOG 10:13:07 am:511 [xhr:GET https://reactnative.dev/movies.json] resolved all response headers to content-type: application/json LOG 10:13:07 am:512 [xhr:GET https://reactnative.dev/movies.json] getResponse (responseType: blob) LOG 10:13:07 am:513 [xhr:GET https://reactnative.dev/movies.json] getResponseHeader Content-Type LOG 10:13:07 am:514 [xhr:GET https://reactnative.dev/movies.json] resolved response header "Content-Type" to application/json LOG 10:13:07 am:515 [xhr] constructed new XMLHttpRequest LOG 10:13:07 am:517 [xhr:GET blob:3b009b36-1a79-47a6-9333-1ccd41fd849b?offset=0&size=0] open GET blob:3b009b36-1a79-47a6-9333-1ccd41fd849b?offset=0&size=0 LOG 10:13:07 am:518 [xhr:GET https://reactnative.dev/movies.json] resolved response Blob (mime type: {"_data":{"blobId":"dcbd9a6b-3501-410b-8475-4f16f6eefd75","offset":0,"size":0,"type":"application/json","__collector":{}}}) application/json LOG 10:13:07 am:518 [xhr:GET https://reactnative.dev/movies.json] found 1 listener(s) for "load" event, calling... LOG 10:13:07 am:519 [xhr:GET https://reactnative.dev/movies.json] getResponse (responseType: blob) LOG 10:13:07 am:520 [xhr:GET https://reactnative.dev/movies.json] getResponseHeader Content-Type LOG 10:13:07 am:520 [xhr:GET https://reactnative.dev/movies.json] resolved response header "Content-Type" to application/json LOG 10:13:07 am:521 [xhr] constructed new XMLHttpRequest LOG 10:13:07 am:522 [xhr:GET blob:cf0693f2-a29f-464c-b945-1ae57ff43832?offset=0&size=0] open GET blob:cf0693f2-a29f-464c-b945-1ae57ff43832?offset=0&size=0 LOG 10:13:07 am:523 [xhr:GET https://reactnative.dev/movies.json] resolved response Blob (mime type: {"_data":{"blobId":"8beb6f91-f2b5-4fb2-a03d-9f77d79fb822","offset":0,"size":0,"type":"application/json","__collector":{}}}) application/json LOG 10:13:07 am:524 [xhr:GET https://reactnative.dev/movies.json] getAllResponseHeaders LOG 10:13:07 am:524 [xhr:GET https://reactnative.dev/movies.json] resolved all response headers to content-type: application/json LOG 10:13:07 am:525 [xhr:GET https://reactnative.dev/movies.json] emitting the "response" event for 1 listener(s)... LOG 10:13:07 am:526 [async-event-emitter:emit] emitting "response" event... LOG 10:13:07 am:527 [async-event-emitter:openListenerQueue] opening "response" listeners queue... LOG 10:13:07 am:528 [async-event-emitter:openListenerQueue] no queue found, creating one... LOG 10:13:07 am:528 [async-event-emitter:emit] appending a one-time cleanup "response" listener... LOG 10:13:07 am:529 [async-event-emitter:emit] emitting "newListener" event... LOG 10:13:07 am:530 [async-event-emitter:openListenerQueue] opening "response" listeners queue... LOG 10:13:07 am:530 [async-event-emitter:openListenerQueue] returning an exising queue: [] LOG 10:13:07 am:531 [async-event-emitter:on] awaiting the "response" listener... LOG 10:13:07 am:532 [async-event-emitter:emit] emitting "removeListener" event... LOG 10:13:07 am:533 [xhr:GET https://reactnative.dev/movies.json] trigger "loadend" {"loaded":0} LOG 10:13:07 am:533 [async-event-emitter:on] "response" listener has resolved! LOG 10:13:07 am:539 [async-event-emitter:emit] cleaned up "request" listeners queue! LOG 10:13:07 am:542 [async-event-emitter:emit] cleaned up "response" listeners queue! ERROR [SyntaxError: JSON Parse error: Unexpected end of input] ```

Expected behavior

Network requests mocked by msw should return the mocked response without errors.

sairus2k commented 11 months ago

After more debugging, it seems the core problem is that response.body is not implemented in React Native. https://github.com/facebook/react-native/issues/27741

I tried using the react-native-polyfill-globals library to polyfill response.body, but msw appears to not mock the requests at all - I just see the actual response from the real API server.

jtdaugh commented 10 months ago

Hitting this issue too. Any progress?

sairus2k commented 10 months ago

Not much. Just made a reproduction without React Native. https://github.com/sairus2k/msw-xhr

jtdaugh commented 10 months ago

Ok. @kettanaito any thoughts here? Happy to contribute to a fix but unclear where to start

sairus2k commented 10 months ago

Ok. @kettanaito any thoughts here? Happy to contribute to a fix but unclear where to start

When discussing reproduction in a non-React Native environment, the issue seems to lie within the Response polyfill. Disabling this polyfill allows the test to pass successfully. We need to either address the issue within the Response polyfill to ensure its compatibility with React Native, or explore alternative polyfills that function correctly in this setting.

kettanaito commented 10 months ago

Can someone clarify why fetch polyfill is even needed? I always thought React Native runs on top of Node.js, so if you are using modern Node.js (v18+), you are getting the global fetch implementation automatically. If you are not, you must, this is a requirement of MSW 2.0.

MSW cannot control how third-party fetch polyfills function, and whether they implement the Fetch API specification faithfully. That's their responsibility. The last time I looked, none of the existing fetch polyfills were faithful to the spec. All sorts of issues may arise because of that.

What we should try next is using undici as the fetch polyfill here. It may not work since it's designed for Node.js and likely depends on Node.js modules that are not present in React Native. A confirmation of this would be nice.


@sairus2k, thanks for putting up that example but it has a few issues in itself:

  1. Fetch globals must not be present in mocks/node. Instead, they must be present in vitest.setup.ts so they are correctly applied before the test environment.
  2. whatwg-fetch mustn't be used. I recommend trying undici instead and see if it works in React Native.
  3. No need to mock global.location if running tests in vitest-environment node. Location will never be used. The same must apply to React Native tests.
sairus2k commented 10 months ago

To clarify the context, we are discussing the approach to mock data requests in iOS simulators and Android emulators. Mocking on real devices might be more complicated than that. And mocking in Jest tests for React Native works just fine.

I always thought React Native runs on top of Node.js, so if you are using modern Node.js (v18+), you are getting the global fetch implementation automatically. If you are not, you must, this is a requirement of MSW 2.0.

The React Native framework does not operate as a Node.js process, as noted in the documentation: React Native JavaScript environment. In more recent versions of React Native, the Hermes JavaScript engine is utilized during runtime In contrast, older versions employ JavaScriptCore, the same engine used by the Webkit browser.

These environments come equipped with global XMLHttpRequest and Fetch implementations.

The Fetch implementation is accessible at this location on GitHub: React Native fetch This is essentially a utilization of whatwg-fetch.

The XMLHttpRequest implementation can be found here: React Native XMLHttpRequest This redirects XMLHttpRequest calls to native libraries on each platform. However, this implementation may not be entirely comprehensive or adhere strictly to specifications. Notably, it lacks support for synchronous calls, and there could be other oversimplifications crucial for MSW.

Can someone clarify why fetch polyfill is even needed?

So, we don't use whatwg-fetch explicitly. React Native uses it implicitly under the hood to align their XMLHttpRequest implementation with the modern Fetch standard.

What we should try next is using undici as the fetch polyfill here. It may not work since it's designed for Node.js and likely depends on Node.js modules that are not present in React Native. A confirmation of this would be nice.

Indeed, undici is tailored for Node.js, distinct from a React Native process. An attempt to use it resulted predictably in errors due to the absence of certain Node.js modules. It is doubtful how it could be beneficial in this scenario.

MSW cannot control how third-party fetch polyfills function, and whether they implement the Fetch API specification faithfully. That's their responsibility. The last time I looked, none of the existing fetch polyfills were faithful to the spec. All sorts of issues may arise because of that.

Identifying the root cause of the issue might necessitate opening an issue in the React Native repository for resolution. Alternatively, a workaround like patch-package could be employed. It's unlikely that MSW itself is the problem.


Thank you for reviewing the repro repository!

  1. Fetch globals must not be present in mocks/node. Instead, they must be present in vitest.setup.ts so they are correctly applied before the test environment.

I moved it to vitest.setup.ts, but it does not seem that makes some difference.

  1. whatwg-fetch mustn't be used. I recommend trying undici instead and see if it works in React Native.

This repository's intent is to emulate the issue as it occurs in React Native, which does utilize whatwg-fetch internally. Thus, it was used here. Unfortunately, undici is incompatible with React Native.

  1. No need to mock global.location if running tests in vitest-environment node. Location will never be used. The same must apply to React Native tests.

I have removed location mock. Previously, there was an error, stating "Cannot read locaiton of undefined". Now the issue is resolved.

kettanaito commented 10 months ago

There isn't much MSW can do until https://github.com/facebook/react-native/issues/27741 is resolved on the React Native side. That would be the proper fix and wouldn't want to do any workarounds to compensate for the lack of specification compliance.

Meanwhile, folks are reporting a different fetch polyfill that can fix the issue. Can somebody please give it a try? Let me know if it works, we could add it to the React Native integration docs.

corentinleberre commented 10 months ago

I tried to use the fetch polyfill that you provided here but unfortunately It don't seem to make a lot of difference..

Is it also a problem with MSW v1.x ?

sairus2k commented 10 months ago

Meanwhile, folks are reporting a different fetch polyfill that can fix the issue. Can somebody please give it a try? Let me know if it works, we could add it to the React Native integration docs.

It works as expected, expect that fact that is not intercepted by MSW. This occurs because the polyfill does not utilize XMLHttpRequest; instead, it employs the native Networking library.

kettanaito commented 10 months ago

It works as expected, expect that fact that is not intercepted by MSW. This occurs because the polyfill does not utilize XMLHttpRequest; instead, it employs the native Networking library.

Oh, I see. This is rather unfortunate. But it also implies that the issue is in the polyfill itself maybe?

Is it also a problem with MSW v1.x ?

Afaik, no, it's not. MSW 1.x doesn't depend on the fetch API primitives as much as 2.0 does, where they become first-class citizens of handling requests/responses in your request handlers.

I will give this issue a bit more time if someone finds out something else about it but so far it looks like it's a candidate for being closed. There's nothing we can/should do on MSW's side to resolve this. It comes down to how requests are polyfilled in React Native.

thomasgrivet commented 9 months ago

Hello! I have the same issue with the native react-native fetch, tried using many polyfills but none of them managed to have the requests intercepted. Decided to use undici but could not get it to work as it depends on node module we don't have. If I understand correctly, the only way to use MSW with react-native as of now is to downgrade to v1?

sairus2k commented 9 months ago

tried using many polyfills but none of them managed to have the requests intercepted

The issue here is that msw/native intercepts only XMLHttpRequest. To fix this, you could modify msw/native to include a Fetch interceptor to try those polyfills. However, I've also encountered the same error while using polyfills.

the only way to use MSW with react-native as of now is to downgrade to v1

I agree, it seems like downgrading to version 1 is the only option for using MSW with React Native at the moment. But, I also had trouble getting it to work properly, even after downgrading.

thomasgrivet commented 9 months ago

Got it working using v1, needed the url polyfill. Only issue is not getting the request headers in my handlers but worked around it by temporarily saving them up when receiving the request:start which somehow does contain said headers 🤷‍♂️.

sairus2k commented 7 months ago

Got it working using patches from this comment https://github.com/mswjs/msw/issues/1926#issuecomment-1905351237

Added those patches to reproduction repo, so now it works.

kettanaito commented 7 months ago

Thank you so much, @sairus2k. The changes to the Interceptors are already merged, will release a new patch version for MSW with this fix once the package is published. Exciting!

kettanaito commented 7 months ago

Released: v2.1.7 🎉

This has been released in v2.1.7!

Make sure to always update to the latest version (npm i msw@latest) to get the newest features and bug fixes.


Predictable release automation by @ossjs/release.

danieloi commented 7 months ago

the current version only works for network calls with the fetch API. Axios, a super popular library, uses XMLHttpRequest and the mocked responses return empty response bodies @kettanaito