IOStream as stdin, stdout and stderr?

[SuperIceCN]

@kawamuray It will be a nice advance if we can use a stream as stdin, stdout and stderr. Currently, I'm trying to transfer the output of my users' wasm program to browser via websocket. This advance will reduce unnecessary io.

[SuperIceCN]

I tried to implemente this, but I encountered a problem. code:

pub struct OutputStream<'a>{
    pub env: &'a JNIEnv<'a>,
    pub stream: &'a JObject<'a>
}

#[async_trait::async_trait]
impl WasiFile for OutputStream<'_>{
    //many lines.....
}

Error:

Compiling wasmtime-jni v0.1.0 (D:\nukkit\wasmtime-java\wasmtime-jni)
error[E0277]: `*mut *const JNINativeInterface_` cannot be shared between threads safely
  --> src\wstream.rs:15:6
   |
15 | impl WasiFile for OutputStream<'_>{
   |      ^^^^^^^^ `*mut *const JNINativeInterface_` cannot be shared between threads safely
   | 
  ::: C:\Users\.cargo\registry\src\mirrors.ustc.edu.cn-61ef6e0cd06fb9b8\wasi-common-0.28.0\src\file.rs:6:28
   |
6  | pub trait WasiFile: Send + Sync {
   |                            ---- required by this bound in `WasiFile`
   |
   = help: within `OutputStream<'impl0>`, the trait `Sync` is not implemented for `*mut *const JNINativeInterface_`
   = note: required because it appears within the type `JNIEnv<'impl0>`
   = note: required because it appears within the type `&'impl0 JNIEnv<'impl0>`
note: required because it appears within the type `OutputStream<'impl0>`
  --> src\wstream.rs:9:12
   |
9  | pub struct OutputStream<'a>{
   |            ^^^^^^^^^^^^

error[E0277]: `*mut _jobject` cannot be shared between threads safely
  --> src\wstream.rs:15:6
   |
15 | impl WasiFile for OutputStream<'_>{
   |      ^^^^^^^^ `*mut _jobject` cannot be shared between threads safely
   | 
  ::: C:\Users\.cargo\registry\src\mirrors.ustc.edu.cn-61ef6e0cd06fb9b8\wasi-common-0.28.0\src\file.rs:6:28
   |
6  | pub trait WasiFile: Send + Sync {
   |                            ---- required by this bound in `WasiFile`
   |
   = help: within `OutputStream<'impl0>`, the trait `Sync` is not implemented for `*mut _jobject`
   = note: required because it appears within the type `jni::objects::JObject<'impl0>`
   = note: required because it appears within the type `&'impl0 jni::objects::JObject<'impl0>`
note: required because it appears within the type `OutputStream<'impl0>`
  --> src\wstream.rs:9:12
   |
9  | pub struct OutputStream<'a>{
   |            ^^^^^^^^^^^^

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0277`.
aborting due to 2 previous errors

error: could not compile `wasmtime-jni`

could not compile `wasmtime-jni`

[kawamuray]

Yeah, because of JObjects are expected to be used in JNI methods locally in their callstack, it doesn't implement Send hence it cannot be passed to any other contexts which potentially be accessed by another thread, like WasiFile. The same situation applies for JNIEnv as well, which is required to call a method of an object.

In order to make a java object movable among threads, we need to create a global reference out of it, also bring around JavaVM instance instead of JNIEnv and attach the executing thread every time it needs to interact with JVM owning items.

I've also looked into some code in wasmtime, and found that instead of implementing WasiFile trait by ourselves, it is easier to use ReadPipe and WritePipe which is provided by wasmtime exactly for this kind of usage - to bridge wasm I/O streams to other language's runtime.

Here's a snippet that I've confirmed it works as expected.

struct JavaOutputStreamWrite {
    jvm: JavaVM,
    obj_ref: GlobalRef,
}

impl JavaOutputStreamWrite {
    fn new(jvm: JavaVM, obj_ref: GlobalRef) -> Self {
        Self { jvm, obj_ref }
    }
}

impl std::io::Write for JavaOutputStreamWrite {
    fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
        // TODO: proper error conversion
        let env = self.jvm.attach_current_thread().unwrap();
        let array = env.byte_array_from_slice(buf).unwrap();
        env.call_method(self.obj_ref.as_obj(), "write", "([B)V", &[array.into()])
            .unwrap();
        Ok(buf.len())
    }

    fn flush(&mut self) -> std::io::Result<()> {
        let env = self.jvm.attach_current_thread().unwrap();
        env.call_method(self.obj_ref.as_obj(), "flush", "()V", &[])
            .unwrap();
        Ok(())
    }
}

    fn native_build(
        env: &JNIEnv,
        _clazz: JClass,
        envs: jobjectArray,
        args: jobjectArray,
        inherit_stdin: jboolean,
        stdin_path: JString,
        inherit_stdout: jboolean,
        stdout_path: JString,
        stdout: JObject,
        inherit_stderr: jboolean,
        stderr_path: JString,
        preopen_dirs: jobjectArray,
    ) -> Result<jlong, Self::Error> {
...
        if inherit_stdout != 0 {
            builder = builder.inherit_stdout();
        } else if !stdout_path.is_null() {
            let file = wasi_utils::open_wasi_file(utils::get_string(env, stdout_path.into())?)?;
            builder = builder.stdout(Box::new(file));
        } else if !stdout.is_null() {
            let jvm = env.get_java_vm()?;
            let global_ref = env.new_global_ref(stdout)?;
            let pipe = WritePipe::new(JavaOutputStreamWrite::new(jvm, global_ref));
            builder = builder.stdout(Box::new(pipe));
        }

    public WasiCtxBuilder stdout(OutputStream out) {
        stdout = out;
        stdoutPath = null;
        inheritStdout = false;
        return this;
    }

[SuperIceCN]

@kawamuray Hello, I encountered a strange problem when I tried to implement stdout:

java.lang.NullPointerException: JNI error: null pointer in get_array_length array argument
    at io.github.kawamuray.wasmtime.wasi.WasiCtxBuilder.nativeBuild(Native Method)
    at io.github.kawamuray.wasmtime.wasi.WasiCtxBuilder.build(WasiCtxBuilder.java:127)
    at io.github.kawamuray.wasmtime.wasi.WasiCtxBuilderTest.testNewConfigWithStdInputStream(WasiCtxBuilderTest.java:66)
    ... <25 internal calls>

my code is here.

[kawamuray]

Can you try spotting which JNIEnv call exactly causing NPE?

btw I saw your code uses byte_array_from_slice for Read implementation as well https://github.com/Superice666/wasmtime-java/blob/4765caef3ff51a5dc31ca14e11ef32d54b385b48/wasmtime-jni/src/wstream.rs#L47 , but I think it doesn't work because it creates a copy array of the given slice to pass to the java rather than making a given slice an actual store of the java array.