search

mswjs / msw

Industry standard API mocking for JavaScript.
https://mswjs.io
MIT License
16.03k stars 525 forks source link

JSON response with special chars truncated since 2.2.0 #2045

Closed LeBenLeBen closed 9 months ago

LeBenLeBen commented 9 months ago

Prerequisites

Environment check

Node.js version

20.9.0

Reproduction repository

https://github.com/LeBenLeBen/msw-truncated

Reproduction steps

Clone the reproduction repo, then:

  1. npm install
  2. npm start
  3. Visit http://localhost:7051/user

Current behavior

The JSON response is truncated a few characters before the end and cannot be parsed by the browser. It’s caused by the special char é in the response. The problem does not occur before MSW 2.2.0, so I’m guessing it’s a regression caused by #1658

Expected behavior

The JSON response should be complete.

ZeroCho commented 9 months ago

I left a comment in #1658 but as this is the same problem with mine, I'll copy it here.

I have an issue with 2.2.0+ version of MSW, when I put CJK(Chinese, Japanese, Korean character) in MSW response. Until msw@2.1.0, it was fine. I checked out what was changed in 2.2.0, and I found this commit could be the problem.

For example, the length of string '조현영' (which is Korean) is 9 bytes in network, whereas 'abc' is 3 bytes. But both are 3 lengths in JavaScript. I think current implementation which calculate length by just string.length can be problem. I think using new TextEncoder().encode(str).length would be better to calculate Content-Length. Thank you

image
kettanaito commented 9 months ago

Root cause

Right now, we are relying on the string length to determine the value of the automatic Content-Length response header.

Instead, we should convert the string to buffer and measure its byte length. There are a couple of ways to do that:

  1. new TextEncoder().encode(text).byteLength
  2. new Blob([text]).size

Pull requests are welcome! I'd start with a failing test for special characters right here:

https://github.com/mswjs/msw/blob/23fc36432a3140043211d142f6dbd63b36efb3e2/src/core/HttpResponse.test.ts#L16

kettanaito commented 9 months ago

Released: v2.2.2 🎉

This has been released in v2.2.2!

Make sure to always update to the latest version (npm i msw@latest) to get the newest features and bug fixes.

Predictable release automation by @ossjs/release.