Choosing a path b/w RustPython and CPython

[bhelx]

Here is where we can have discussion on where to go with this repo

There are two paths I'm exploring:

• Use RustPython
  • This is what the repo is currently using
  • It was easy to get compiling
  • stuck on an error in the interpreter
  • further stuck debugging because when compiling in development mode there are too many locals in the interpret loop.

• Compile CPython 3.11 to wasm, maybe adapt SingleStore/python-wasi

  • Had some trouble getting it to compile. Looked at a few different places
  • Need some more time to work through the compilation issues or maybe backport to an a working version
  • Would we need to write any kind of rust bindings for this?

  I think using CPython would be preferred but I'm open to anything that gets us started.

[ubiquitous-dev]

Fermyon is using the python-wasi build. I think there are some core differences in their approach vs Extism though:

• They use the lowest possible denominator of files and env vars
• WAGI is built on top of WASI

Their runtime has no need to have a native understanding of the context it's running in, like it does with Extism, because their plugins are just making env var or stdin/stdout/fs calls (and relying on python-wasi converting those into wasi commands for wasmtime to interpret).

This does mean that they're severely limited, in that they can't provide additional functionality like Host Functions, dynamic configuration, http, or logging, that Extism currently provides.

I think some key attributes of deciding which runtime to use are:

• What is the overall performance of the compiled program? is there an order-of-magnitude difference between the 2?
• How large is the build?
• Are there any other limitations on the Python side?
• How complex and brittle is the process of overriding the global space to expose the additional PDK methods?

[atifsyedali]

Any updates?

[bhelx]

I haven't looked at it. I'm still getting the same problem with rust-python, and i still feel like that project is not the best way forward regardless.

There has been quite of bit of progress with cpython with regards to wasm so I was going to look at this again before 1.0: https://github.com/extism/extism/issues/420

[bhelx]

FWIW, to follow up here. I've made some good progress with cpython and I feel like that's a more sustainable solution. I'm going to close this issue in favor of #2 . I'll be adding updates there and in our #python-pdk channel in Discord.