search

fermyon / spin-python-sdk

Spin SDK for Python
https://developer.fermyon.com/spin/v2/python-components
Apache License 2.0
30 stars 11 forks source link

No module named `zlib` #24

Open radu-matei opened 1 year ago

radu-matei commented 1 year ago

I think zlib is part of the standard library — https://docs.python.org/3/library/zlib.html.

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: ModuleNotFoundError: No module named 'zlib'', crates/spin-python-engine/src/lib.rs:385:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: the `wizer.initialize` function trapped

Caused by:
    wasm trap: wasm `unreachable` instruction executed
    wasm backtrace:
        0: 0x33f727 - <unknown>!__rust_start_panic
    note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable to may show more debugging information

Error: Couldn't create wasm from input
jasonwashburn commented 1 year ago

I played around with this briefly, trying out some related "fixes" from other python WASM compiled projects, but didn't make any real headway. Willing to keep poking at it as I have time since it seems to be the biggest thing preventing pip installation of a number of common packages I tried, but to prevent reinventing the wheel, just wondering if anyone has any existing thoughts or theories on what the problem might be or potential fixes that haven't been explored fully?

dicej commented 1 year ago

Looks like zlib is a native module which is conditionally compiled into CPython only if the upstream zlib C library is detected at configuration time: https://github.com/python/cpython/blob/a3a5b4bb232ba29875147baa3b56a5524eece353/PC/config.c#L58-L60. I'm guessing that's not being found when we build CPython for WASI, and thus the module is not included.

I haven't dug any deeper than that yet, but my guess is we need to supply a libz.a (built using wasi-sdk) and tell the CPython configuration machinery where to find it.

jasonwashburn commented 1 year ago

Ok, that was my take on it as well, I did try building/linking it to libz that I'd previously compiled using the wasi-sdk without effect previously, albeit it was fairly late one night and I wasn't being particularly systematic about my approach at the time, so its probably worth attempting again in a more controlled matter. I'll see if I can figure anything out.

radu-matei commented 1 year ago

Update: https://github.com/python/cpython/issues/91246#issuecomment-1603957637