Bun.stdin.stream() and Bun.file(0).stream() consistently hang on 983036 bytes of 1048576 bytes expected

[guest271314]

What version of Bun is running?

v1.1.12 i

What platform is your computer?

x64

What steps can reproduce the bug?

Follow these instructions https://github.com/guest271314/native-messaging-bun/tree/main

Installation and usage on Chrome and Chromium

1. Navigate to chrome://extensions.
2. Toggle Developer mode.
3. Click Load unpacked.
4. Select native-messaging-bun folder.
5. Note the generated extension ID.
6. Open nm_bun.json in a text editor, set "path" to absolute path of nm_bun.js and chrome-extension://<ID>/ using ID from 5 in "allowed_origins" array.
7. Copy the file to Chrome or Chromium configuration folder, e.g., Chromium on *nix ~/.config/chromium/NativeMessagingHosts; Chrome dev channel on *nix ~/.config/google-chrome-unstable/NativeMessagingHosts.
8. Make sure bun executable and nm_bun.js are executable.
9. To test click service worker link in panel of unpacked extension which is DevTools for background.js in MV3 ServiceWorker, observe echo'ed message from Bun Native Messaging host. To disconnect run port.disconnect().

The Native Messaging host echoes back the message passed.

Point to this Native Messaging host https://github.com/guest271314/NativeMessagingHosts/blob/4878edcb14401d3f2b5aac08448f8e0f0916956a/nm_host.js in nm_bun.js where all that is needed is the shebang line

#!/usr/bin/env -S /home/user/bin/bun run /home/user/bin/nm_host.js

/*
#!/usr/bin/env -S /home/user/bin/deno run -A /home/user/bin/nm_host.js
#!/usr/bin/env -S /home/user/bin/node --experimental-default-type=module /home/user/bin/nm_host.js
#!/usr/bin/env -S /home/user/bin/bun run --smol /home/user/bin/nm_host.js
*/

const runtime = navigator.userAgent;
const buffer = new ArrayBuffer(0, { maxByteLength: 1024 ** 2 });
const view = new DataView(buffer);
const encoder = new TextEncoder();
const { dirname, filename, url } = import.meta;

let readable, writable, exit, args;

if (runtime.startsWith("Deno")) {
  ({ readable } = Deno.stdin);
  ({ writable } = Deno.stdout);
  ({ exit } = Deno);
  ({ args } = Deno);
}

if (runtime.startsWith("Node")) {
  const { Duplex } = await import("node:stream");
  ({ readable } = Duplex.toWeb(process.stdin));
  ({ writable } = Duplex.toWeb(process.stdout));
  ({ exit } = process);
  ({ argv: args } = process);
}

if (runtime.startsWith("Bun")) {
  readable = Bun.stdin.stream();
  writable = new WritableStream({
    async write(value) {
      await Bun.write(Bun.stdout, value);
    },
  }, new CountQueuingStrategy({ highWaterMark: Infinity }));
  ({ exit } = process);
  ({ argv: args } = Bun);
}

function encodeMessage(message) {
  return encoder.encode(JSON.stringify(message));
}

async function* getMessage() {
  let messageLength = 0;
  let readOffset = 0;
  for await (let message of readable) {
    if (buffer.byteLength === 0) {
      buffer.resize(4);
      for (let i = 0; i < 4; i++) {
        view.setUint8(i, message[i]);
      }
      messageLength = view.getUint32(0, true);
      message = message.subarray(4);
      buffer.resize(0);
    }
    buffer.resize(buffer.byteLength + message.length);
    for (let i = 0; i < message.length; i++, readOffset++) {
      view.setUint8(readOffset, message[i]);
    }
    if (buffer.byteLength === messageLength) {
      yield new Uint8Array(buffer);
      messageLength = 0;
      readOffset = 0;
      buffer.resize(0);
    }
  }
}

async function sendMessage(message) {
  await new Blob([
    new Uint8Array(new Uint32Array([message.length]).buffer),
    message,
  ])
    .stream()
    .pipeTo(writable, { preventClose: true });
}

try {
  await sendMessage(encodeMessage([{ dirname, filename, url }, ...args]));
  for await (const message of getMessage()) {
    await sendMessage(message);
  }
} catch (e) {
  exit();
}

/*
export {
  args,
  encodeMessage,
  exit,
  getMessage,
  readable,
  sendMessage,
  writable,
};
*/

What is the expected behavior?

Bun Native Messaging host to echo back an Array having length 209715 based on https://github.com/guest271314/native-messaging-bun/blob/main/background.js#L6C1-L6C37

port.postMessage(new Array(209715));

The same code run in Bun, Deno, and Node.js runtime environments worked as of this commit https://github.com/guest271314/NativeMessagingHosts/commit/4878edcb14401d3f2b5aac08448f8e0f0916956

What do you see instead?

The Array is never sent back to the Native Messaging client (Chromium 127 Developer Build).

Additional information

The same code still works nm_host.js still works using deno version 1.44. There are three (3) DevTools windows open in the screenshot; Bun Native Messaging extension window is on top, then Deno then Node.js. Notice the windows underneath have arrays with length at 209715 where the Bun host does not send back the message to the client and will exit when sending a sencond message and node version v23.0.0-nightly20240526aaca18b54e. Deno and Node.js handle subsequent messages.

[guest271314]

Looks like Bun only reads either a certain number of times or a certain amount of data from Bun.stdin.stream().

Or, the resizable ArrayBuffer implementation has an internal limit that is under 1 MB.

For the same input of 209715 length array encoded as JSON (1 MB) Node.js and Deno each read 16 times and the entire input stream. Bun reads 5 times and stops short of reading the entire input, thus never echoes back the Array(209715) to the client.

[guest271314]

Bun.file("/dev/stdin").stream() and Bun.file("/proc/self/fd/0").stream() achieve the expected result.

Bun.stdin.stream() and Bun.file(0).stream() do not ahcieve the expected result, consistently hang on 983036 bytes of 1048576 bytes expected.

Bun.stdin.stream() does not behave the same as Bun.file().

Re-reading the guides https://bun.sh/guides/process/stdin it's not clear if Bun.file() and Bun.stdin are intended to be synonymous.

Bun also exposes stdin as a BunFile via Bun.stdin. This is useful for incrementally reading large inputs that are piped into the bun process.