Doesn't work for compressed responses

[Maxim-Mazurok]

Hello, first of all, thank you for creating this, I'm using it to integrate with 3rd-party API, and it helps me to ensure that my mocks/snapshots will stay up to date with the actual 3rd-party API that I don't control.

The issue I'm having is that when the actual server replies with compressed response (brotli in my case) - it looks like node-fetch tries to decompress the response, and fails because it's already a plain text (as replaced/transformed by this plugin).

This is the error message I'm getting:

FetchError: Invalid response body while trying to fetch https://bla.com/bla: 
Decompression failed
 ❯ consumeBody node_modules/node-fetch/src/body.js:235:60
 ❯ Response.text node_modules/node-fetch/src/body.js:158:18
 ❯ Module.getBla util.ts:42:19
     40|   console.log(response);
     41| 
     42|   const result = (await response.text());
       |                   ^
     43| 
     44|   console.log(result);
 ❯ util.integration.spec.ts:6:20

I have tried to remove some headers to help node-fetch understand that it's plain text:

var transform = function (snapshot) {
    return function (res) {
        res.status = snapshot.response.status;
        res.statusText = snapshot.response.statusText;
        res.body = snapshot.response.bodyText;
        snapshot.response.headers.forEach(function (_a) {
            var k = _a[0], v = _a[1];
            res.headers.append(k, v);
        });
+       res.headers.delete("content-encoding");
+       res.headers.delete("content-length");
        return res;
    };
};

But it didn't help.

For now, I'll be using this workaround to disable compression for requests:

const response = await fetch(
  url.toString(),
  {
    headers: {
      "accept-encoding": "identity", // need to disable compression because `msw-snapshot` doesn't seem to support it
    },
  },
);

Let me know what you think, happy to help with this!

[hrsh7th]

I think your solution is correct.

But the identify is depending on the server implementation. (The server still can return compressed response if specified the identify.

Could you test with "Accept-Encoding": "gzip"?

If it worked, I think it's better solution because gzip is very popular encoding.

[hrsh7th]

Or, Could you know the free, open and public API that able to reproduce this problem?

[Maxim-Mazurok]

gzip doesn't seem to work either, giving incorrect header check, even though API seems to support it and snapshot looks fine.

 FAIL  util.spec.ts > getHoneyProductHistory > works
FetchError: Invalid response body while trying to fetch https://d.joinhoney.com/v3?operationName=web_getProductPriceHistory&variables=%7B%22productId%22%3A%227613592105936880680_c9cfbae01a9af7d396c1f977224a4b8a_c9cfbae01a9af7d396c1f977224a4b8a%22%2C%22timeframe%22%3A30%7D: 
incorrect header check
 ❯ consumeBody node_modules/node-fetch/src/body.js:234:60
 ❯ Response.text node_modules/node-fetch/src/body.js:158:18
 ❯ Response.json node_modules/node-fetch/src/body.js:148:16
 ❯ Module.getHoneyProductHistory util.ts:45:19
     43|   );
     44| 
     45|   const result = (await response.json()) as GetProductPriceHistoryAPI;
       |                   ^
     46| 
     47|   return result;
 ❯ util.spec.ts:36:20

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯Serialized Error: {
  "code": "Z_DATA_ERROR",
  "errno": "Z_DATA_ERROR",
  "erroredSysCall": undefined,
}
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯

I'm not sure if that API is public, but it works without any special headers/cookies so one could use it for testing compression: https://d.joinhoney.com/v3?operationName=web_getProductPriceHistory&variables=%7B%22productId%22%3A%227613592105936880680_c9cfbae01a9af7d396c1f977224a4b8a_c9cfbae01a9af7d396c1f977224a4b8a%22%2C%22timeframe%22%3A30%7D

[hrsh7th]

Hm... Unfortunately, I can't reproduce the provided API with adding it as a new test-case.

My environment are

• Node.js@18.12.1
• node-fetch@v2.6.7 (modified for this reproduction)
• MacOS

[hrsh7th]

@Maxim-Mazurok I think the v0.2.0 solves this problem. Could you try it?

[hrsh7th]

I've re-published the v0.3.0.

[Maxim-Mazurok]

I've tried v1.1.0 and it works, thank you!