Piping stdout results in output truncated to 64kb

[vergenzt]

If I redirect stdout from madge directly to a file then it works just fine. However whenever I pipe madge's stdout to another program (in my case bcomps, to try to find some dead code), for some reason madge's output gets truncated to 65536 bytes.

Example:

$ npx madge --dot src > redirect.dot
$ npx madge --dot src | tee pipe.dot >/dev/null
$ wc -c *.dot | head -n-1
 65536 pipe.dot
260141 redirect.dot

For thoroughness, I confirmed that the first 65535 bytes of the non-truncated result are indeed equal to the truncated result:

$ md5sum *.dot
16310f635d12d5585ed5d28341f54621  pipe.dot
cb1bcef7cfecd8ce278b35bd4b7aa4d0  redirect.dot

$ md5sum <(head -c65536 redirect.dot)
16310f635d12d5585ed5d28341f54621  /dev/fd/63

If I understand correctly, I believe that's equal to the size of my (MacOS) system's pipe buffer in this case:

How big is the pipe buffer? - Unix & Linux Stack Exchange

Mac OS X, for example, uses a capacity of 16384 bytes by default, but can switch to 65336 byte capacities if large write are made to the pipe ...

Could this be related to https://github.com/nodejs/node/issues/22088? Given the CLI's output mechanism appears to be simply instantiating a promise without awaiting: https://github.com/pahen/madge/blob/c12ef70836855df7b9c34e5d93dfd34fdcae230d/bin/cli.js#L151

My theory: When writing directly to a file, none of the I/O promises down the chain ever have to block, so they're able to resolve to completion without Node exiting. However, once a pipe is involved, the pipe can block output until there's space on the buffer. When Node tries to write to the full pipe the write fails, and Node.js has to wait to retry. Because Node.js sees nothing on the event loop, it exits?

(That doesn't fully make sense to me because I'd think there would be something on the event loop for the retry... 🤔 Maybe it never actually retries failed writes? Not sure...)

---

Version info:

$ madge --version          # => 5.0.1
$ node --version           # => v19.2.0
$ npm --version            # => 8.19.3
$ sw_vers -productVersion  # => 12.6

[togakangaroo]

Still happening on v8 btw. The fact that its exactly 2^16 characters and hence the exact max pipe size doesn't make sense to me with the theory, but maybe I'm missing something.

[togakangaroo]

The reason this is breaking is because on linux and only when stdout is the pipe, node's stream.write() call is async.

Sam tracked down the why here

In the case of madge, simply not calling process.exit(0) on success would fix the issue

[hiroshi-ishii]

Seeing this also on macOS 14.7.1 with node 18.20.4 and Madge 8.0.0