What are the possible values that can be passed to Javy.IO.readSync()?

[guest271314]

What is your question?

I'm testing compiling JavaScript source code to a Native Messaging host in WASM form using javy and wasmtime to execute the compiled .wasm file.

It doesn't look like the compiled code is reading stdin from Chromium browser.

What file descriptor does Javy.IO.readSync() read from?

[guest271314]

Let's say the use case is to read standard input from a different PID, is that possible using Javy.IO.readSync()?

[jeffcharles]

For the moment, we just support reading from file descriptor 0 in the Wasm instance which normally corresponds with stdin. That said, file descriptor 0 in the Wasm instance will read from whatever stream you have your WASI context configured to read from in the host environment. In the absence of a WASI context configured to provide a stream for file descriptor 0, the Wasm instance will be able to call Javy.IO.readSync but won't receive any bytes.

It doesn't look like the compiled code is reading stdin from Chromium browser.

Do you mind sharing how you're setting up your WASI context in the browser?

Let's say the use case is to read standard input from a different PID, is that possible using Javy.IO.readSync()?

Not in the Wasm instance, only 0 is supported. But you can map file descriptor 0 to arbitrary streams in the host when setting up the WASI context.

[guest271314]

whatever stream you have your WASI context configured to read from in the host environment

I'm not not sure what this means in code.

Do you mind sharing how you're setting up your WASI context in the browser?

To create a Native Messaging host (see https://github.com/guest271314/NativeMessagingHosts?tab=readme-ov-file#native-messaging-documentation) I compile JavaScript source with javy compile nm_javy.js -o nm_javy.wasm, optionally optimaize with wasmtime compile --optimize opt-level=s nm_javy.wasm to generate a .cwasm, which is greater size file, yet faster than .wasm in my tests to read and write 1 MB of JSON (1045876 bytes) https://github.com/guest271314/native-messaging-webassembly/blob/main/nm_javy.js

const stdin = 0;
const stdout = 1;

function encodeMessage(str) {
  return new TextEncoder().encode(JSON.stringify(str));
}

function getMessage() {
  let offset = 0;
  const header = new Uint8Array(4);
  Javy.IO.readSync(stdin, header);
  const [length] = new Uint32Array(header.buffer);
  const message = new Uint8Array(length);
  while (1) {
    const buffer = new Uint8Array(1024);
    const bytesRead = Javy.IO.readSync(stdin, buffer);
    message.set(buffer.subarray(0, bytesRead), offset);
    offset += bytesRead;
    if (offset === length) {
      break;
    }
  }
  return message;
}

function sendMessage(json) {
  let header = Uint32Array.from({
    length: 4,
  }, (_, index) => (json.length >> (index * 8)) & 0xff);
  let output = new Uint8Array(header.length + json.length);
  output.set(header, 0);
  output.set(json, 4);
  Javy.IO.writeSync(stdout, output);
}

function main() {
  while (1) {
    try {
      const message = getMessage();
      sendMessage(message);
    } catch (e) {
      sendMessage(encodeMessage(e.message));
    }
  }
}

main();

In the shell script

wasmtime compile --optimize opt-level=s nm_javy.wasm

The reason I inquire about different file descriptors is due to how I have to read standard input stream in Google V8's d8 shell and Amazon Web Services - Labs LLRT, because neither engine or runtime processes stdin to an ArrayBuffer. In the case of d8 the input is executed, so I get the PID of the current process, the read from /proc/<PID>/fd/0 and send the stdin to d8 back to d8 using GNU Coreutils head, or dd command, or QuickJS in a subprocess https://github.com/guest271314/NativeMessagingHosts/blob/main/nm_d8.js#L39-L116

function main() {
  // Get PID of current process
  const pid = (os.system("pgrep", ["-n", "-f", os.d8Path])).replace(/\D+/g, "")
    .trim();
  // Get PPID of current process
  const ppid = os.system("ps", ["-o", "ppid=", "-p", JSON.parse(pid)]);
  // readline() doesn't read past the Uint32Array equivalent message length
  // V8 authors are not interested in reading STDIN to an ArrayBuffer in d8
  // https://groups.google.com/g/v8-users/c/NsnStT6bx3Y/m/Yr_Z1FwgAQAJ
  // Use dd https://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html
  // or GNU Coreutils head https://www.gnu.org/software/coreutils/manual/html_node/head-invocation.html
  // or QuickJS https://bellard.org/quickjs/
  const bash = ["bash", [
    "--posix",
    "-c",
    `
length=$(head -q -z --bytes=4 /proc/${pid}/fd/0 | od -An -td4 -)
message=$(head -q -z --bytes=$((length)) /proc/${pid}/fd/0)
# length=$(dd iflag=fullblock oflag=nocache conv=notrunc,fdatasync bs=4 count=1 if=/proc/${pid}/fd/0 | od -An -td4 -)
# message=$(dd iflag=fullblock oflag=nocache conv=notrunc,fdatasync bs=$((length)) count=1 if=/proc/${pid}/fd/0)
printf "$message"
`,
  ]];
  const qjs = ["/home/user/bin/qjs", [
    "--std",
    "-m",
    "-e",
    `const path = "/proc/${pid}/fd/0";
  try {
    const size = new Uint32Array(1);
    const err = { errno: 0 };
    const pipe = std.open(
      path,
      "rb",
      err,
    );
    if (err.errno !== 0) {
      throw std.strerror(err.errno);
    }
    pipe.read(size.buffer, 0, 4);
    // writeFile("len.txt", size);
    // {error: 'writeFile is not defined'
    const output = new Uint8Array(size[0]);
    pipe.read(output.buffer, 0, size[0]);
    const res = new Uint8Array([...new Uint8Array(size.buffer),...output]);
    std.out.write(res.buffer, 0, res.length);
    std.out.flush();
    std.exit(0);
  } catch (e) {
    const json = JSON.stringify({error:e.message});
    std.out.write(Uint32Array.of(json.length).buffer, 0, 4);
    std.out.puts(json);
    std.out.flush();
    std.exit(0);
  }
`,
  ]];
  while (true) {
    // Terminate current process when chrome processes close
    if (!(os.system("pgrep", ["-P", JSON.parse(ppid)]))) {
      break;
    }
    const message = getMessage(
      pid,
      qjs,
    );
    if (message) {
      sendMessage(message);
      if (
        // Handle error from qjs
        String.fromCodePoint.apply(null, [...message.subarray(1, 8)]) ===
          `"error"` // JSON
      ) {
        break;
      }
    }
  }
}

I use the same pattern for llrt https://github.com/guest271314/NativeMessagingHosts/blob/main/nm_d8.js#L39-L116, which also doesn't really provide a way to read standard input stream to an ArrayBuffer with built-in interfaces

async function main() {
  const pid = process.id;
  const bash = ["bash", [
    "--posix",
    "-c",
    `
length=$(head -q -z --bytes=4 /proc/${pid}/fd/0 | od -An -td4 -)
message=$(head -q -z --bytes=$((length)) /proc/${pid}/fd/0)
# length=$(dd iflag=fullblock oflag=nocache conv=notrunc,fdatasync bs=4 count=1 if=/proc/${pid}/fd/0 | od -An -td4 -)
# message=$(dd iflag=fullblock oflag=nocache conv=notrunc,fdatasync bs=$((length)) count=1 if=/proc/${pid}/fd/0)
printf "$message"
`,
  ]];
  const qjs = ["/path/to/qjs", [
    "--std",
    "-m",
    "-e",
    `const path = "/proc/${pid}/fd/0";
  try { 

I should be able to read specific file descriptors given the advancements in WASI interfaces.

Moreover, since JavaScript in ECMA-262 does not specify reading standard input streams and writing strandard output streams at all, I am thinking of using WebAssembly to read stdin and write to stdout using the same code across multiple JavaScript/WebAssembly engines and runtimes, to manifest one form of a "universal executable".

In this case, I am asking for clarification if we can just write "/proc/<PID>/fd/0" instead of 0 to Javy.IO.readSync()?

[jeffcharles]

whatever stream you have your WASI context configured to read from in the host environment

I'm not not sure what this means in code.

Concretely, what implementation in the host are you using to satisfy the Javy module's wasi_snapshot_preview1 fd_read import? Something must be satisfying that import or your Javy module would fail to instantiate due to unmet imports.

As an example, here's a link to Rust code that configures WASI. What's the equivalent to:

    let wasi = WasiCtxBuilder::new()
        .inherit_stdio()
        .inherit_args()?
        .build();

in your codebase?

I should be able to read specific file descriptors given the advancements in WASI interfaces.

The APIs exist in WASI but they won't return non-empty values unless your WASI implementation is configured appropriately.

In this case, I am asking for clarification if we can just write "/proc//fd/0" instead of 0 to Javy.IO.readSync()?

No. Javy.IO.readSync expects a file descriptor. Right now only 0 is supported. /proc/<PID>/fd/0 is a path, not a file descriptor. Hypothetically we could add an equivalent to POSIX's fopen which would take a file path as an argument and then return a file descriptor to that file which could then be passed to Javy.IO.readSync (the file descriptor would be an integer but not 0).

But what I'd recommend doing is configuring your WASI implementation on the host to provide the contents of a buffer as the result to return for fd_read for file descriptor 0 (fd_read is the WASI call Javy.IO.readSync makes under the covers) instead of trying to read the stdin of the host process.

E.g., the equivalent of

        let input = &[...]; // the value you want in stdin
        let wasi = WasiCtxBuilder::new()
            .stdin(Box::new(ReadPipe::from(input)))
            .build();

[guest271314]

Hypothetically we could add an equivalent to POSIX's fopen which would take a file path as an argument and then return a file descriptor to that file which could then be passed to Javy.IO.readSync (the file descriptor would be an integer but not 0).

That's what I am asking about.

But what I'd recommend doing is configuring your WASI implementation on the host to provide the contents of a buffer as the result to return for fd_read for file descriptor 0 (fd_read is the WASI call Javy.IO.readSync makes under the covers) instead of trying to read the stdin of the host process.

I'm not following.

[jeffcharles]

Hypothetically we could add an equivalent to POSIX's fopen which would take a file path as an argument and then return a file descriptor to that file which could then be passed to Javy.IO.readSync (the file descriptor would be an integer but not 0).

That's what I am asking about.

I'd like to dig more into why fd 0 (in the guest) doesn't work for your use case. My understanding is you can't use an inherited fd 0 from your host process because your host process doesn't have the input to read available on stdin. Is that correct?

But what I'd recommend doing is configuring your WASI implementation on the host to provide the contents of a buffer as the result to return for fd_read for file descriptor 0 (fd_read is the WASI call Javy.IO.readSync makes under the covers) instead of trying to read the stdin of the host process.

I'm not following.

Reading from stdin (fd 0) inside the Javy module does not read from stdin (fd 0) of the host process. They're completely separate unless the host process has configured its WASI runtime to inherit stdin.

As a concrete example:

Let's say I have a JS file that I compile with Javy to index.wasm that contains:

const buffer = new Uint8Array(1024);
Javy.IO.readSync(0, buffer);
const decoded = new TextDecoder().decode(buffer);
console.log(decoded);

That is it reads up to 1024 bytes from stdin writes to stderr.

And I have the following text file called input.txt:

hi from the file!

And I have the following Rust program:

use std::fs;

use wasi_common::{pipe::ReadPipe, sync::WasiCtxBuilder};
use wasmtime::*;

fn main() -> Result<()> {
    let engine = Engine::default();
    let module = Module::from_file(&engine, "index.wasm")?;

    // Use a static input
    let my_input = b"hi from static input";
    {
        let mut linker = Linker::new(&engine);
        wasi_common::sync::add_to_linker(&mut linker, |s| s)?;

        let wasi = WasiCtxBuilder::new()
            // Where we use a static input for stdin
            .stdin(Box::new(ReadPipe::from(my_input.as_ref())))
            .inherit_stderr()
            .build();

        let mut store = Store::new(&engine, wasi);
        linker.module(&mut store, "", &module)?;
        linker
            .get_default(&mut store, "")?
            .typed::<(), ()>(&store)?
            .call(&mut store, ())?;
    }

    // Use a file as input
    let my_file = "input.txt";
    {
        let mut linker = Linker::new(&engine);
        wasi_common::sync::add_to_linker(&mut linker, |s| s)?;

        let wasi = WasiCtxBuilder::new()
            // Where we use a file as stdin
            .stdin(Box::new(ReadPipe::from(fs::read(my_file)?)))
            .inherit_stderr()
            .build();

        let mut store = Store::new(&engine, wasi);
        linker.module(&mut store, "", &module)?;
        linker
            .get_default(&mut store, "")?
            .typed::<(), ()>(&store)?
            .call(&mut store, ())?;
    }

    Ok(())
}

The program above will run the Javy module and first use it's own stdin, then use the contents of a buffer, and then use the contents of a file to provide values for stdin for the Javy module.

If you run the program above:

$ echo "hi from stdin" | cargo run
   Compiling javy-question v0.1.0 (/Users/jeffcharles/projects/javy-question)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.38s
     Running `target/debug/javy-question`
hi from static input
hi from the file!

You'll notice for the same module that Javy does not use the host process's stdin but instead uses the contents of an array and then the contents of an arbitrary file for what it reads from stdin. This is accomplished by configuring the WASI context.

[guest271314]

Your example appears to use hardcoded scripting or compiled code.

Here's an example using QuickJS directly that works because I can pass the dynamic PID of the Native Messaging host process to the dynamically evaluated script passed as a parameter to the subprocess

  const qjs = ["/home/user/bin/qjs", [
    "--std",
    "-m",
    "-e",
    `const path = "/proc/${pid}/fd/0";
  try {
    const size = new Uint32Array(1);
    const err = { errno: 0 };
    const pipe = std.open(
      path,
      "rb",
      err,
    );
    if (err.errno !== 0) {
      throw std.strerror(err.errno);
    }
    pipe.read(size.buffer, 0, 4);
    // writeFile("len.txt", size);
    // {error: 'writeFile is not defined'
    const output = new Uint8Array(size[0]);
    pipe.read(output.buffer, 0, size[0]);
    const res = new Uint8Array([...new Uint8Array(size.buffer),...output]);
    std.out.write(res.buffer, 0, res.length);
    std.out.flush();
    std.exit(0);
  } catch (e) {
    const json = JSON.stringify({error:e.message});
    std.out.write(Uint32Array.of(json.length).buffer, 0, 4);
    std.out.puts(json);
    std.out.flush();
    std.exit(0);
  }
`,
  ]];

Executing with wasmtime when the wasm file is not a subprocess the script successfully reads fd 0.

When executed as subprocess does not read fd 0 of the parent process

  const wasm = ["/home/user/bin/wasmtime", [
    "-C",
    "cache=n",
    "--allow-precompiled",
    "/home/user/localscripts/native-messaging-webassembly/nm_javy.cwasm"
  ]];
  while (true) {
    // Terminate current process when chrome processes close
    if (!(os.system("pgrep", ["-P", JSON.parse(ppid)]))) {
      break;
    }
    const message = getMessage(
      pid,
      wasm,
    );

It should be possible to pass the dynamically generated PID of the current process to the WASM subprocess, defined in source JavaScript code.

You can reproduce the above by following the instructions here https://github.com/guest271314/native-messaging-d8, compiling JavaScript to WASM (or JavaScript passed to embedded QuickJS, however you look at it) using this souce code https://github.com/guest271314/native-messaging-webassembly/blob/main/nm_javy.js, and for completeness, though not necessary to reproduce, optimize with wasmtime compile --optimize opt-level=s nm_javy.wasm. Then use wasmtime and the `.wasm or .cwasm file to read standard input to Google V8's d8 shell as a subprocess executed using os.system(), and then read stdin passed back to d8 from WASM.

We could write this is Rust, though QuickJS support this out of the box with std.open().

Works with QuickJS directly, and using dd command, or GNU head, because we can dynamically pass /proc/<PID>/fd/0 to the subprocess.

We should be able to do that either dynamically compiling the JavaScript source code to write in the hardcoded PID; or, ideally, just be able to dynamically pass the parameters to the WASM module.

[jeffcharles]

So if I'm understanding correctly, you're using QuickJS to spawn a Wasmtime subprocess with os.exec and then using that Wasmtime subprocess to execute the Javy module.

Did you try something along the lines of this when invoking Wasmtime from QuickJS:

import * as os from 'os';

const pid = 1234; // substitute with what PID you want
const inputFd = os.open(`/proc/${pid}/fd/0`);
os.exec(["wasmtime", "index.wasm"], { stdin: inputFd });
os.close(inputFd);

Also you can do something like this from QuickJS if you want to have a string for input (you can do something similar with an array but the method calls are a little trickier):

import * as os from 'os';
import * as std from 'std';

const input = "Input to the Javy module";

const inputToUseAsStdin = std.tmpfile();
inputToUseAsStdin.puts(input);
inputToUseAsStdin.flush();
inputToUseAsStdin.seek(0);
os.exec(["wasmtime", "index.wasm"], { stdin: inputToUseAsStdin.fileno() });
inputToUseAsStdin.close();

If I save that to quickjs-script.js and run it, I see:

$ ~/src/github.com/bellard/quickjs/qjs quickjs-script.js
Input to the Javy module

From what I'm reading above, it sounds like you would also need APIs to read command line arguments or environment variables to be able to receive the PID or whatever input you want to pass through and the APIs for reading command line arguments or environment variables aren't available in Javy. As well, Wasmtime's binary is not going to let a Wasm module read from the host filesystem without additional configuration.

For example, given this C program called main.c that tries to read a file:

#include <stdio.h>
#include <stdlib.h>

int main() {
    FILE *file = fopen("my_file.txt", "r");
    if (file == NULL) {
        perror("Error opening text file");
        return 1;
    }

    char ch;
    while ((ch = fgetc(file)) != EOF) {
        putchar(ch);
    }

    fclose(file);
    return 0;
}

You'll see:

$ ~/wasi-sdk-22.0/bin/clang main.c -o main.wasm
$ clang main.c                                 
$./a.out
My file content
$ wasmtime main.wasm
Error opening text file: No such file or directory

due to the restrictions in place on the Wasm sandbox.

[guest271314]

So if I'm understanding correctly, you're using QuickJS to spawn a Wasmtime subprocess with os.exec and then using that Wasmtime subprocess to execute the Javy module.

No.

I am creating Native Messaging https://developer.chrome.com/docs/extensions/mv3/nativeMessaging/ hosts for engines, runtimes, programming languages.

In the case of Google V8's d8 shell, stdin is expected to be from a TTY, and does not really provide a way to read "32-bit message length in native byte order" followed by JSON message

Chrome starts each native messaging host in a separate process and communicates with it using standard input (stdin) and standard output (stdout). The same format is used to send messages in both directions; each message is serialized using JSON, UTF-8 encoded and is preceded with 32-bit message length in native byte order.

See https://groups.google.com/g/v8-users/c/NsnStT6bx3Y/m/Yr_Z1FwgAQAJ. And readline() and ReadFromStdin() https://github.com/v8/v8/blob/e6c4e2484b8de2678e12cbb0973009dce341b206/src/d8/d8.cc#L4370C1-L4373C2.

So, what I am doing in the case for creating a Native Messaging host for V8's d8, where os.system() is exposed, is executing qjs - not WASM, the actual qjs executable - to read stdin to d8 that Chromium browser sends, returning the value from qjs to d8.

The way I do that is by dynamically passing the PID of the d8 process to qjs, which reads /proc/<PID>/fd/0, referencing the d8 process stdin.

What I am wondering is can Javy be configured to read that dynamic PID. Basically just to do this with WASM, for a universal executable to read stdin to various programs, dynamically. We should be able to pass parameters to WASM functions.

due to the restrictions in place on the Wasm sandbox.

If WASM can read /dev/stdin, or specifically /proc/PID/fd/0 of it's own PID, it should be able to read aby PID, given wasi-io, wasi-cli, et al.

[jeffcharles]

I am creating Native Messaging https://developer.chrome.com/docs/extensions/mv3/nativeMessaging/ hosts for engines, runtimes, programming languages.

Okay, I'm still lost. What's the host application in this case? Is it a program you've written yourself? Or are you using the Wasmtime CLI as the host application?

I'm also not clear on where d8 fits into this. Is that the host application?

If WASM can read /dev/stdin, or specifically /proc/PID/fd/0 of it's own PID, it should be able to read aby PID, given wasi-io, wasi-cli, et al.

The APIs are certainly available to do that. But the process hosting the Wasm instance needs to explicitly grant the Wasm instance access. The Wasmtime CLI automatically does that specifically for stdin, stdout, and stderr but not for other files or file descriptors.

We should be able to pass parameters to WASM functions.

You can pass numbers (specifically signed 32-bit and 64-bit integers and 32-bit and 64-bit floats). That's all the Wasm specification supports. Passing arrays (and by extension strings and other data types) requires writing data into the Wasm instance's memory from the host process and then passing the address as the offset into linear memory and the length of the data as two integers. That's what the WASI calls do under the covers. Luckily the PID is a number so you should be fine there if all you want to pass is a PID.

[jeffcharles]

Actually I'm now realizing we currently disallow function parameters in exported Wasm functions from Javy due to not having any logic present to convert Wasm numbers to JavaScript numbers in the QuickJS runtime. So either that would have to be added or we would need to expose JavaScript APIs to read environment variables or command line parameters (and the hosting process would need to provide access to those env vars or command line parameters).

[guest271314]

Okay, I'm still lost. What's the host application in this case? Is it a program you've written yourself? Or are you using the Wasmtime CLI as the host application?

If you use Chromium-based browsers (Chromium, Chrome, Brave, Edge, Opera), or use Firefox browser, including Nightly, or you use Safari browser - you can launch any arbitrary local application from the browser, for example, this very page both of us are typing on.

The host in this case is that arbitrary application. For example, I have written Native Messaging hosts in Python, C, C++, Bash, WebAssembly using C compiled to WASM with WASI-SDK, Rust, txiki.js, QuickJSm Node.js, Deno, Bun, Amazon Web Services - Labs LLRT, Google V8's d8 shell, Mozilla's SpiderMonkey js shell. And now, Bytecode Alliance's Javy, your gear.

In the case of d8, it's a very basic V8 JavaScript/WebAssembly engine shell - not intended to read binary data from stdin, rather is designed as a REPL. So to read the standard input to d8 from the browser, I use d8's os.system() call to launch QuickJS as a subprocess to read the input from the browser, then, still inside the d8 shell instance, read the output from qjs, then proceed as I would with other Native Messaging hosts to do the work.

Whether that work be capturing entire system audio in the application and streaming to the browser; launching a full-duplex connection using Deno or Node.js's fetch(), controlled from, streamed to and from the browser; streaming speech synthesis input and output. Basically whatever is not generally possible in the browser using Web API's alone.

The base case is reading stdin in the application, which is the message length, essentially as a uint32_t or Uint32Array, the maximum length of JSON message from the host to browser being 1 MB at a time, then the message itself, JSON, generally encoded as a Uint8Array, depending on the IO implementation of the programming language or application.

In the case of Javy, we have exactly one (1) option, 0, refering exclusively to the WASM result of Javy compilation, hardcoded.

QuickJS's qjs is ~ 1MB, as I'm sure you know, and is able to achieve the requirement. Since Javy uses an embedded QuickJS, the capability to read from any file descriptor should exist in the QuickJS source code you are using.

I have also read standard input streams to the host application using GNU Coreutils head, or the dd command, then returned that stadard input stream to the original running script it was intended to go, but couldn't get there due to lack of IO support to read the Native Messaging protocol

each message is serialized using JSON, UTF-8 encoded and is preceded with 32-bit message length in native byte order.

So, I was thinking of testing WASM for this case.

And moreover, since JavaScript itself does not specifiy IO, using the WASM result from Javy as a "universal executable", one of the original ideals of WebAssembly, as I understand it.

If that's not possible, I can just keep using QuickJS directly. It's probably, well, I know for sure, it's less expensive than using wasmtime and javy.

[jeffcharles]

I have written Native Messaging hosts in [..] WebAssembly using C compiled to WASM with WASI-SDK, [..]

Can you go into more detail around how you're executing the WebAssembly? I don't know much about Native Messaging, but I'm guessing it can't instantiate and execute Wasm modules without going through an application installed on the host computer.

QuickJS's qjs is ~ 1MB, as I'm sure you know, and is able to achieve the requirement. Since Javy uses an embedded QuickJS, the capability to read from any file descriptor should exist in the QuickJS source code you are using.

That capability exists in the quickjs-libc portion of the Quickjs codebase but we never compiled that part to Wasm (and neither does rquickjs which is what we now use). That part (meaning the whole quickjs-libc.h and quickjs-libc.c files) of the code includes logic for system calls which are not supported by WASI.

The logic backing Javy.IO.readSync is provided by Rust code we've authored. And to add similar support for other I/O operations, we'd have to author and maintain more Rust code to do that.

And moreover, since JavaScript itself does not specifiy IO, using the WASM result from Javy as a "universal executable", one of the original ideals of WebAssembly, as I understand it.

I'm not entirely following what you mean by WASM result. Exported Javy functions don't have return values (at least for now) due to not having the code to map a return value expressed as QuickJS variable to one or more Wasm values. Javy functions can return data through stdout and stderr streams (file descriptors 1 and 2).

The short suggestion I'd have to work around what you're running into is, substitute whatever program you're instantiating and executing the WebAssembly modules with (I'm guessing the Wasmtime CLI but I don't know your architecture), with a different program that you write where you embed Wasmtime in it as a library so you can handle mapping the file streams however you would like when instantiating and running the Wasm module.

[guest271314]

Can you go into more detail around how you're executing the WebAssembly?

Sure.

You can run the code yourself to become familiar with what Native Messaging protocol is. Just like I had to dive in to WASI-SDK to gather what's going on in WASI world.

Like this https://github.com/guest271314/native-messaging-webassembly/blob/main/nm_c_wasm.sh

#!/usr/bin/env -S /path/to/wasmtime -C cache=n /path/to/nm_c.wasm

this https://github.com/guest271314/native-messaging-webassembly/blob/main/nm_c_wat.sh

#!/bin/bash
# https://www.reddit.com/r/bash/comments/10i09se/comment/j5blsrw/
# https://github.com/bytecodealliance/wasmtime/issues/3715#issuecomment-1399308863

script='
(module
  (type (;0;) (func (param i32 i32 i32) (result i32)))
  (type (;1;) (func (param i32 i64 i32) (result i64)))
  (type (;2;) (func (param i32) (result i32)))
  (type (;3;) (func (param i32 i32) (result i32)))
  (type (;4;) (func (param i32 i32 i32 i32) (result i32)))
  (type (;5;) (func (param i32 i64 i32 i32) (result i32)))
  (type (;6;) (func (param i32)))
  ;; ...
  (data $.data (i32.const 1032) "\09\00\00\00\00\00\00\00\00\00\00\00\01\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\02\00\00\00\00\00\00\00\03\00\00\00\08\07\00\00\00\04\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\08\04\00\00\00\00\00\00\05\00\00\00\00\00\00\00\00\00\00\00\01\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\05\00\00\00\03\00\00\00\18\0b\00\00\00\04\00\00\00\00\00\00\00\00\00\00\01\00\00\00\00\00\00\00\0a\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\00\80\04\00\00"))

'

./kill_wasmtime.sh &
./wasmtime <(printf '%s' "$script")

and this https://github.com/guest271314/native-messaging-webassembly/blob/main/nm_javy_wasm.sh

#!/usr/bin/env -S /path/to/wasmtime -C cache=n --allow-precompiled /path/to/nm_javy.wasm

The logic backing Javy.IO.readSync is provided by Rust code we've authored. And to add similar support for other I/O operations, we'd have to author and maintain more Rust code to do that.

Whatever you have to do. QuickJS, Rust. Doesn't really matter to me.

I'm not entirely following what you mean by WASM result.

The .wasm file spit out by javy.

Javy functions can return data through stdout and stderr streams (file descriptors 1 and 2).

Right. I'm just requesting that be extended to any arbitrary file descriptor.

The short suggestion I'd have to work around what you're running into is, substitute whatever program you're instantiating and executing the WebAssembly modules with (I'm guessing the Wasmtime CLI but I don't know your architecture), with a different program that you write where you embed Wasmtime in it as a library so you can handle mapping the file streams however you would like when instantiating and running the Wasm module.

This is basically a feature request now, since you are not now exposing anything other than the file descriptor associated with the current process.

To understand what is going on you might want to try following the instructions here https://github.com/guest271314/native-messaging-webassembly/tree/main or here https://github.com/guest271314/NativeMessagingHosts?tab=readme-ov-file#installation-and-usage to create a Native Messaging host.

If you are not interested in extending Javy to read from arbitrary file descriptors and don't plan on actually testing installing a Native Messaging host, then I can close this out now.

[guest271314]

@jeffcharles Thanks for trying to figure out my question, BTW.

[jeffcharles]

Happy to help!

I'll just offer one final follow-up. File descriptor 0 in a Wasm instance is not the same as file descriptor 0 in the process. The process can configure the Wasm instance's WASI context so they are the same. That's what the Wasmtime CLI does out of the box (as you've seen). But you can write your own program that embeds Wasmtime as a library and in that program, you can change what file descriptor 0 in the Wasm instance reads from to a different file or file descriptor or something else entirely.

[guest271314]

I'll have to look in to that. I don't think I've seen an example of WASM/WASI reading arbitrary file descriptors.

It's hard to beat QuickJS at ~1MB for the capability out of the box

const path = "/proc/${pid}/fd/0";
  try {
    const size = new Uint32Array(1);
    const err = { errno: 0 };
    const pipe = std.open(
      path,
      "rb",
      err,
    );