node-fetch polyfill causes indefinitely pending promises when using the retry middleware

[NetroScript]

Hey, I am working on a NodeJS project (using TypeScript) and I had the following code roughly:

const data = await wretch(url)
        .middlewares([
          retry()
        ])
        .get()
        .json();

And having the middleware in there causes an infinitely pending await.

At first I thought I did something wrong, and tried different things.

For that I changed it to the following:

const response = await wretch(url)
  .middlewares([
    retry()
  ])
  .get()
  .res();

let data;
try {
  data = await response.json();
} catch (e) {
  console.log(e);
}

This now correctly returns a response object, where I can see that the request was successful.

But as soon as I call .json() I once again have a indefinitely hanging promise. The same is true for other Promises based resolvers for the body like .text().

I setup the polyfills for NodeJS as described on the main repository.

So I had the idea that it might be caused by the node-fetch polyfill. So I compiled the Typescript files, and ran it with the flag --experimental-fetch (Node version 17.5.0) and indeed, it works flawlessly when using the built-in fetch function.

But the problem is, in my project setup using this experimental flag is rather awkward and I would prefer to not use it, while keeping the middleware in there.

So my final question is:

Do you have any idea how to fix this (/circumvent it). Maybe a different resolver can already give me what I want, but so far I didn't get anything working which I wanted to work.

---

I want to mention two more things:

The delay middleware works without any issues.

Also the request I am doing does not do a single retry, but is successful in the first try. I also didn't consume the Request in any way.

[elbywan]

Hey @NetroScript,

I think that the issue could be related to the way node-fetch implements response cloning which is used under the hood by the retry middleware.

Could you try to increase the highWaterMark as described in their documentation?

https://github.com/node-fetch/node-fetch#custom-highwatermark

[NetroScript]

This does indeed fix the problem.

But should someone in the future also have the same problem I will also describe the solution I now have:

I am using TypeScript and I am using commonjs. This means I couldn't import node-fetch@3 the default way (due to it not working with require in the TS output) and was just using version 2 before. (But version 3 is needed as only starting with that version highWaterMark was added).

To be able to use version 3 with my setup, I had to use the following code in my async bootstrapping function:

async function bootstrap() {
  // If NodeJS supports the fetch function, we do not need to import it
  if (typeof fetch == 'undefined') {
    const _importDynamic = new Function(
      'modulePath',
      'return import(modulePath)',
    );
    const { default: fetch } = await _importDynamic('node-fetch');
    global.fetch = fetch;
  }
  ....
}

bootstrap();

And the final function looking like this works without problems:

    const data = await wretch(
      url,
      {
        highWaterMark: 1024 * 1024,
      },
    )
      .middlewares([retry()])
      .get()
      .json();