npm / minipass-fetch

An implementation of window.fetch in Node.js using Minipass streams
Other
54 stars 11 forks source link

fix: do not allow back pressure to slow receiving data from the socket #8

Closed nlf closed 4 years ago

nlf commented 4 years ago

The previous behavior here pipes the socket directly into an instance of minipass, which applies back pressure through the entire pipeline of streams all the way down to the filesystem. When the filesystem or cpu become slow, this increased back pressure slows the rate at which we consume data from the socket to the point that we end up with the socket getting into a state where we are no longer receiving data from it and I believe it's closing on its own.

This change ignores the back pressure from the socket and instead forcibly writes all the data it receives directly to this first instance of minipass. Applying back pressure from there is safe since we've already consumed the data from the socket and no longer need to rely on it being open and actively sending us data.

This seems to resolve quite a few of our cb() never called errors in the npm cli