Using wazero.NewCompilationCache() as a saved variable doesnt work across NewPlugin invocations

[apisorbust]

I thought it might be possible to share the compilation cache across plugin instantiations. In my case I am creating one plugin instance per .wasm module. So maybe my error is assuming multiple .wasm modules can share the same cache? None the less I tried using a variable to store a single instance of cache:

var compilationCache := wazero.NewCompilationCache()

then using that variable when creating plugin instances:

defer func(cache wazero.CompilationCache, ctx context.Context) {
        err := cache.Close(ctx)
        if err != nil {

        }
    }(compilationCache, ctx)

    config := extism.PluginConfig{
        EnableWasi:    true,
        ModuleConfig:  wazero.NewModuleConfig(),
        RuntimeConfig: wazero.NewRuntimeConfig().WithCompilationCache(compilationCache),
        LogLevel:      e.logLevel,
    }

That would end up causing a Panic:

panic: assignment to entry in nil map

goroutine 1 [running]:
github.com/tetratelabs/wazero/internal/engine/wazevo.(*engine).addCompiledModuleToMemory(0xc0000f44d0, 0xc000509180, 0xc0002402d0)
        C:/Users/kevin/go/pkg/mod/github.com/tetratelabs/wazero@v1.7.1/internal/engine/wazevo/engine_cache.go:84 +0x93
github.com/tetratelabs/wazero/internal/engine/wazevo.(*engine).addCompiledModule(0xc0000f44d0, 0xc000509180, 0xc0002402d0)
        C:/Users/kevin/go/pkg/mod/github.com/tetratelabs/wazero@v1.7.1/internal/engine/wazevo/engine_cache.go:39 +0x25
github.com/tetratelabs/wazero/internal/engine/wazevo.(*engine).CompileModule(0xc0000f44d0, {0x82cf78, 0xb6b6a0}, 0xc000509180, {0x0, 0x0, 0x0}, 0x0)
        C:/Users/kevin/go/pkg/mod/github.com/tetratelabs/wazero@v1.7.1/internal/engine/wazevo/engine.go:143 +0xd1
github.com/tetratelabs/wazero.(*runtime).CompileModule(0xc00002cff0, {0x82cf78, 0xb6b6a0}, {0xaa5c00, 0xdb4, 0xdb4})
        C:/Users/kevin/go/pkg/mod/github.com/tetratelabs/wazero@v1.7.1/runtime.go:237 +0x206
github.com/tetratelabs/wazero.(*runtime).InstantiateWithConfig(0xc00002cff0, {0x82cf78, 0xb6b6a0}, {0xaa5c00?, 0x0?, 0x0?}, {0x831a18, 0xc0002400f0})
        C:/Users/kevin/go/pkg/mod/github.com/tetratelabs/wazero@v1.7.1/runtime.go:273 +0x38
github.com/extism/go-sdk.NewPlugin({0x82cf78, 0xb6b6a0}, {{0xc0000322b0, 0x1, 0x1}, 0x0, 0x0, {0x0, 0x0, 0x0}, ...}, ...)
        C:/Users/kevin/go/pkg/mod/github.com/extism/go-sdk@v1.2.0/extism.go:339 +0x3f2

SO.. perhaps this caching ONLY works for creating multiple instances of the SAME plugin? Not a big deal either way.. just thought perhaps this would save a little bit of resources across new plugin instantiations.

[bhelx]

I think maybe you are misunderstanding what the compilation cache is caching. It’s caching the compiled module, not the instance. To cache an instance you probably want to look at the plug-in pool PR

[bhelx]

Following back up from discussion in Discord.

Please give this a try:

ctx := context.Background()
cache := wazero.NewCompilationCache()
defer cache.Close(ctx)

manifest := Manifest{Wasm: []Wasm{WasmFile{Path: "wasm/noop.wasm"}}}

config := PluginConfig{
    EnableWasi:    true,
    ModuleConfig:  wazero.NewModuleConfig(),
    RuntimeConfig: wazero.NewRuntimeConfig().WithCompilationCache(cache),
}

plugin1, err := NewPlugin(ctx, manifest, config, []HostFunction{})
plugin2, err := NewPlugin(ctx, manifest, config, []HostFunction{})

[bhelx]

Any update as to whether this works or not?

[mhmd-azeez]

Hello @apisorbust I have tested this code (which uses the cache for 2 different plugins) and it seems to work. I am closing the issue, but feel free to reopen it if it doesn't work for you, or if that doesn't exactly cover what you meant

func main() {
    ctx := context.Background()
    compilationCache := wazero.NewCompilationCache()

    defer func(cache wazero.CompilationCache, ctx context.Context) {
        err := cache.Close(ctx)
        if err != nil {
            fmt.Printf("Failed to close compilation cache: %v\n", err)
            os.Exit(1)
        }
    }(compilationCache, ctx)

    manifest1 := extism.Manifest{
        Wasm: []extism.Wasm{
            extism.WasmUrl{
                Url: "https://github.com/extism/plugins/releases/latest/download/count_vowels.wasm",
            },
        },
    }

    manifest2 := extism.Manifest{
        Wasm: []extism.Wasm{
            extism.WasmUrl{
                Url: "https://github.com/extism/plugins/releases/latest/download/consume.wasm",
            },
        },
    }

    config1 := extism.PluginConfig{
        EnableWasi:    true,
        ModuleConfig:  wazero.NewModuleConfig(),
        RuntimeConfig: wazero.NewRuntimeConfig().WithCompilationCache(compilationCache),
    }
    _, err := extism.NewPlugin(ctx, manifest1, config1, []extism.HostFunction{})

    if err != nil {
        fmt.Printf("Failed to initialize plugin 1: %v\n", err)
        os.Exit(1)
    }

    config2 := extism.PluginConfig{
        EnableWasi:    true,
        ModuleConfig:  wazero.NewModuleConfig(),
        RuntimeConfig: wazero.NewRuntimeConfig().WithCompilationCache(compilationCache),
    }
    _, err = extism.NewPlugin(ctx, manifest2, config2, []extism.HostFunction{})

    if err != nil {
        fmt.Printf("Failed to initialize plugin 2: %v\n", err)
        os.Exit(1)
    }

    fmt.Println("Successfully initialized plugins")
}

Result:

Successfully initialized plugins