Imports (Host Functions) example not functioning

[ryn9]

I have having trouble getting the "Imports (Host Functions)" example to function.

main.go (using go v1.21.6 and go-pdk v1.0.1)

package main

import (
    "github.com/extism/go-pdk"
)

func main() {}

//go:wasmimport extism:host/user a_python_func
func aPythonFunc(uint64) uint64

//export hello_from_python
func helloFromPython() int32 {
    msg := "An argument to send to Python"
    mem := pdk.AllocateString(msg)
    defer mem.Free()
    ptr := aPythonFunc(mem.Offset())
    rmem := pdk.FindMemory(ptr)
    response := string(rmem.ReadBytes())
    pdk.OutputString(response)
    return 0
}

compiled with tinygo (0.30.0): tinygo build -target=wasi -o plugin.wasm main.go

app.py (extism 1.0.1 installed via pip)

from extism import host_fn, Function, ValType, Plugin

@host_fn
def a_python_func(plugin, input_, output, _user_data):
    # The plug-in is passing us a string
    input_str = plugin.input_string(input_[0])

    # just printing this out to prove we're in Python land
    print("Hello from Python!")

    # let's just add "!" to the input string
    # but you could imagine here we could add some
    # applicaiton code like query or manipulate the database
    # or our application APIs
    input_str += "!"
# 
    # set the new string as the return value to the plug-in
    plugin.return_string(output[0], input_str)

functions = [
    Function(
        "a_python_func",
        [ValType.I64],
        [ValType.I64],
        a_python_func,
        None
    )
]

manifest = {"wasm": [{"path": "./plugin.wasm"}]}
plugin = Plugin(manifest, functions=functions, wasi=True)
#result = plugin.call('hello_from_python').decode('utf-8') <- function name has a typo
result = plugin.call('helloFromPython').decode('utf-8')
print(result)

Executing results in the following error:

$ python app.py
Traceback (most recent call last):
  File "app.py", line 21, in <module>
    Function(
  File "/home/user/extism_wasm_test/lib/python3.8/site-packages/extism/extism.py", line 254, in __init__
    self.pointer = _lib.extism_function_new(
TypeError: initializer for ctype 'void(*)(struct ExtismCurrentPlugin *, ExtismVal *, unsigned long, ExtismVal *, unsigned long, void *)' must be a cdata pointer, not function

[nilslice]

Hi @ryn9 - could you upload the plugin code to https://modsurfer.dylibso.com and share the link so we can take a look at what you're running?

result = plugin.call('hello_from_python').decode('utf-8') <- function name has a typo

Based on this code:

//export hello_from_python
func helloFromPython() int32 { ... }

The export should be called hello_from_python so its strange that you had to call it with the name its declared with in Go.

[ryn9]

@nilslice I will try to get to that in about an hour. Everything though, is just a copy from the example with the one line I guess mistakingly commented out and replaced with another line. I'm pretty sure there's just something wrong with the python module though from what the error reads.

[ryn9]

@nilslice modsurfer url from the plugin: https://modsurfer.dylibso.com/module?hash=010f52d00c090967921f35663d75e76151f78044a7d607451480c3c7d49a98ce

The WASM was created from this source code:

package main

import (
    "github.com/extism/go-pdk"
)

func main() {}

//go:wasmimport extism:host/user a_python_func
func aPythonFunc(uint64) uint64

//export hello_from_python
func helloFromPython() int32 {
    msg := "An argument to send to Python"
    mem := pdk.AllocateString(msg)
    defer mem.Free()
    ptr := aPythonFunc(mem.Offset())
    rmem := pdk.FindMemory(ptr)
    response := string(rmem.ReadBytes())
    pdk.OutputString(response)
    return 0
}

Trying to execute with the following

(extism_wasm_test) user@host:~/extism_wasm_test$ sha256sum plugin.wasm
010f52d00c090967921f35663d75e76151f78044a7d607451480c3c7d49a98ce  plugin.wasm

(extism_wasm_test) user@host:~/extism_wasm_test$ cat app.py

from extism import host_fn, Function, ValType, Plugin

@host_fn
def a_python_func(plugin, input_, output, _user_data):
    # The plug-in is passing us a string
    input_str = plugin.input_string(input_[0])

    # just printing this out to prove we're in Python land
    print("Hello from Python!")

    # let's just add "!" to the input string
    # but you could imagine here we could add some
    # applicaiton code like query or manipulate the database
    # or our application APIs
    input_str += "!"

    # set the new string as the return value to the plug-in
    plugin.return_string(output[0], input_str)

functions = [
  Function(
      "a_python_func",
      [ValType.I64],
      [ValType.I64],
      a_python_func,
      None
  )
]

manifest = {"wasm": [{"path": "./plugin.wasm"}]}
plugin = Plugin(manifest, functions=functions, wasi=True)
result = plugin.call('hello_from_python').decode('utf-8')
print(result)

(extism_wasm_test) user@host:~/extism_wasm_test$ python3 app.py
Traceback (most recent call last):
  File "app.py", line 21, in <module>
    Function(
  File "/home/user/extism_wasm_test/lib/python3.8/site-packages/extism/extism.py", line 254, in __init__
    self.pointer = _lib.extism_function_new(
TypeError: initializer for ctype 'void(*)(struct ExtismCurrentPlugin *, ExtismVal *, unsigned long, ExtismVal *, unsigned long, void *)' must be a cdata pointer, not function

I am out of my league here, but I don't believe this has anything to do with the WASM plugin.

I am pretty sure there is something wrong with the provided sample python, or an issue with the python SDK.

[nilslice]

Yea based on the modsurfer link, it appears to all line up -- and the error you shared is definitely originating in the Python SDK.. perhaps @zshipko, @bhelx, @chrisdickinson who are more familiar with it could take a look.

Out of curiosity, have you tried with a more recent version of python than 3.8?

[chrisdickinson]

Ah, I think I see the problem – in Extism 1.0 we updated the host_fn decorator to accept arguments. If you have a typed host function, you should be able to write something like:

from extism import host_fn, Function, ValType, Plugin

@host_fn()
def a_python_func(input_str: str, *_user_data) -> str:
    # just printing this out to prove we're in Python land
    print("Hello from Python!")
    # set the new string as the return value to the plug-in
    return input_str + "!"

manifest = {"wasm": [{"path": "./plugin.wasm"}]}
plugin = Plugin(manifest, functions=[a_python_func], wasi=True)
result = plugin.call('count_vowels', (13).to_bytes(length=4)).decode('utf-8')
print(result)

An untyped version would look like this:

from extism import host_fn, Function, ValType, Plugin

@host_fn(signature=([ValType.I64], [ValType.I64]))
def host_reflect(plugin, input_, output):
    # The plug-in is passing us a string
    input_str = plugin.input_string(input_[0])

    # just printing this out to prove we're in Python land
    print("Hello from Python!")

    # let's just add "!" to the input string
    # but you could imagine here we could add some
    # applicaiton code like query or manipulate the database
    # or our application APIs
    input_str += "!"

    # set the new string as the return value to the plug-in
    plugin.return_string(output[0], input_str)

# NB: I changed this to use our example "reflect" plugin
manifest = {"wasm": [{"url": "https://github.com/extism/plugins/releases/download/v1.0.0-rc2/reflect.wasm"}]}
plugin = Plugin(manifest, functions=[host_reflect], wasi=True)
result = plugin.call('reflect', b'hello world').decode('utf-8')
print(result)

[ryn9]

I will give it a try in a short bit. If you wouldn't mind updating the docs (I think some of the other SDK docs may need to be updated as well)...

[ryn9]

Thank you for your assistance! I can verify the second ran... I will have to play with the first one later..

[nilslice]

Thank you! We'll go through and update the readmes - sorry for that.

[ryn9]

Late, but I have have tested the updated docs, and they look good :) Thank you!