wasmtime bindgen `get_terminal_stdin` NotImplementedError when using WIT component

ConradMearns commented 6 months ago

I'm attempting to follow along with https://github.com/bytecodealliance/componentize-py to write a python WASM component that returns "Hello World!", and import that WASM module into python to execute it.

I have a hello.wit

package example:hello;

world hello {
  export hello: func() -> string;
}

hello.py

import hello
class Hello(hello.Hello):
    def hello(self) -> str:
        return "Hello world!"

I can build the WASM file with

componentize-py -d hello.wit hello bindings .
componentize-py -d hello.wit -w hello componentize app -o hello.wasm

but when I try to start the process of utilizing this module, I run into an error with wasmtime.bindgen

poetry run python -m wasmtime.bindgen hello.wasm --out-dir hellowasm

Traceback (most recent call last):
  File "/home/conrad/test_wasmwit/.venv/lib/python3.11/site-packages/wasmtime/_func.py", line 260, in enter_wasm
    yield byref(trap)
  File "/home/conrad/test_wasmwit/.venv/lib/python3.11/site-packages/wasmtime/_func.py", line 101, in __call__
    raise WasmtimeError._from_ptr(error)
wasmtime._error.WasmtimeError: error while executing at wasm backtrace:
    0:  0x100 - wit-component:shim!indirect-wasi:cli/terminal-stdin-get-terminal-stdin
    1: 0x254f - wit-component:adapter:wasi_snapshot_preview1!_ZN22wasi_snapshot_preview111descriptors11Descriptors3new17ha2fdde51d30a71bdE
    2: 0x1640 - wit-component:adapter:wasi_snapshot_preview1!_ZN22wasi_snapshot_preview15State11descriptors17h780c546b61bcfc7fE
    3: 0x180c - wit-component:adapter:wasi_snapshot_preview1!fd_write
    4:  0x13a - wit-component:shim!adapt-wasi_snapshot_preview1-fd_write
    5: 0x24c210 - <unknown>!_ZN4wasi13lib_generated8fd_write17hd4964fea612b930fE
    6: 0x2487fd - <unknown>!_ZN80_$LT$std..io..Write..write_fmt..Adapter$LT$T$GT$$u20$as$u20$core..fmt..Write$GT$9write_str17h375f1d6863bea9dfE
    7: 0x25622e - <unknown>!_ZN4core3fmt5write17h0eddb54b80b97b9dE
    8: 0x24971f - <unknown>!_ZN3std2io5Write9write_fmt17h6d46415105134b08E
    9: 0x24b4a5 - <unknown>!_ZN3std9panicking12default_hook28_$u7b$$u7b$closure$u7d$$u7d$17he04c18047097e21eE
   10: 0x248ddb - <unknown>!_ZN3std9panicking12default_hook17hb03d7fae0dedb715E
   11: 0x24ba7e - <unknown>!_ZN3std9panicking20rust_panic_with_hook17hc93abff18edee779E
   12: 0x24b141 - <unknown>!_ZN3std9panicking19begin_panic_handler28_$u7b$$u7b$closure$u7d$$u7d$17h922bcdd9c6fdedfbE
   13: 0x24b0a6 - <unknown>!_ZN3std10sys_common9backtrace26__rust_end_short_backtrace17h2597d6ecb1d3419eE
   14: 0x24b72a - <unknown>!rust_begin_unwind
   15: 0x25108c - <unknown>!_ZN4core9panicking9panic_fmt17h35d9e7e9c02f9eb5E
   16: 0x251662 - <unknown>!_ZN4core9panicking5panic17h2d50353119445d1cE
   17: 0xa107 - <unknown>!_ZN7bindgen7bindgen18InterfaceGenerator5types17h97d459020892fdb1E
   18: 0x4963 - <unknown>!_ZN7bindgen7bindgen10WasmtimePy8generate17hb9172d8bba7255b9E
   19: 0x3161d - <unknown>!_ZN78_$LT$bindgen..bindings..PythonBindings$u20$as$u20$bindgen..bindings..Guest$GT$8generate17h087b694eee28f7aeE
   20: 0x31826 - <unknown>!generate

Caused by:
    python exception

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/conrad/test_wasmwit/.venv/lib/python3.11/site-packages/wasmtime/bindgen/__main__.py", line 40, in <module>
    main()
  File "/home/conrad/test_wasmwit/.venv/lib/python3.11/site-packages/wasmtime/bindgen/__main__.py", line 30, in main
    files = generate(name, contents)
            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/conrad/test_wasmwit/.venv/lib/python3.11/site-packages/wasmtime/bindgen/__init__.py", line 144, in generate
    result = root.generate(store, name, component)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/conrad/test_wasmwit/.venv/lib/python3.11/site-packages/wasmtime/bindgen/generated/__init__.py", line 288, in generate
    ret = self.lift_callee0(caller, ptr, len0, ptr1, len2)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/conrad/test_wasmwit/.venv/lib/python3.11/site-packages/wasmtime/_func.py", line 91, in __call__
    with enter_wasm(store) as trap:
  File "/nix/store/5k91mg4qjylxbfvrv748smfh51ppjq0g-python3-3.11.6/lib/python3.11/contextlib.py", line 155, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/conrad/test_wasmwit/.venv/lib/python3.11/site-packages/wasmtime/_func.py", line 266, in enter_wasm
    maybe_raise_last_exn()
  File "/home/conrad/test_wasmwit/.venv/lib/python3.11/site-packages/wasmtime/_func.py", line 276, in maybe_raise_last_exn
    raise exn
  File "/home/conrad/test_wasmwit/.venv/lib/python3.11/site-packages/wasmtime/_func.py", line 184, in trampoline
    pyresults = func(*pyparams)
                ^^^^^^^^^^^^^^^
  File "/home/conrad/test_wasmwit/.venv/lib/python3.11/site-packages/wasmtime/bindgen/generated/__init__.py", line 200, in lowering16_callee
    ret = import_object.terminal_stdin.get_terminal_stdin()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/conrad/test_wasmwit/.venv/lib/python3.11/site-packages/wasmtime/bindgen/__init__.py", line 95, in get_terminal_stdin
    raise NotImplementedError
NotImplementedError

My versions from poetry show

Python 3.11.6
componentize-py           0.9.2     Tool to package Python applications as WebAssembly components
wasmtime                  15.0.0    A WebAssembly runtime powered by Wasmtime

I'm not sure if I'm maybe missing a step, or when the breaking change was introduced.

If I find a version combination that works I'll report back :)

alexcrichton commented 5 months ago

Ah I think this is a bug in the bindings generator. Would be good to have a fix, but I alas don't have time to take a look at this at this time.

XavierGeerinck commented 4 months ago

+1 on this, do you maybe have had some time to look into this already @alexcrichton ? Is there any other way of getting componentized Python modules to work with wasmtime?

alexcrichton commented 4 months ago

Alas I still do not have time to look into this, but assistance in helping to fix issues in the bindings would be much appreciated!

dicej commented 3 months ago

To clarify the issue here: componentize-py does not currently have a way to stub out or omit wasi:cli imports required by the CPython interpreter and standard library, and since wasmtime-py does not include a wasi:cli implementation, it can't run componentize-py-generated components. Very early versions of componentize-py did have such a feature, which is why it worked at one point, but the addition of native extension support required a large refactor, and I was forced to drop that feature.

I do plan to revisit that and add an option to stub out WASI imports in componentize-py, but it would have to be done carefully. For example, we might need to patch the random module so it (A) doesn't try to use wasi:random to initialize the PRNG seed and (B) returns an error if any application code tries to use it.

dicej commented 3 months ago

BTW, there are two things missing in wasmtime-py that prevent it from running components with WASI 0.2 imports (whether by way of componentize-py or some other tool):

No compatible WASI 0.2 implementation (i.e. there's no way to reuse the wasmtime-wasi Rust implementation)
No resource support

Both of those items would require significant effort.

dicej commented 3 months ago

I went ahead and dug into this on the componentize-py side, adding a --stub-wasi feature which may be used to stub-out any WASI imports. This has the side effect of baking in the PRNG seed in Python's random module, so it should not be used for apps which need a access to a secure RNG. Here's an example: https://github.com/bytecodealliance/componentize-py/tree/main/examples/sandbox

bytecodealliance / wasmtime-py

wasmtime bindgen `get_terminal_stdin` NotImplementedError when using WIT component #202