extism / go-sdk

Extism Go SDK - easily run WebAssembly modules in your Go applications
https://pkg.go.dev/github.com/extism/go-sdk
BSD 3-Clause "New" or "Revised" License
85 stars 10 forks source link

chore: bump wazero version #64

Closed nilslice closed 5 months ago

nilslice commented 8 months ago

Opening this up as a draft for now for discussion.

This PR brings wazero up to the latest version, which introduces their new optimizing compiler. I've added benchmarks from the current version we use v1.3.0, and the latest v1.7.0:

v1.3.0:

goos: linux
goarch: amd64
pkg: github.com/extism/go-sdk
cpu: 12th Gen Intel(R) Core(TM) i7-1255U
BenchmarkInitialize/noop-12                  314           3610336 ns/op         2850623 B/op       2744 allocs/op
BenchmarkInitializeWithCache/noop-12                2060            549867 ns/op         1630131 B/op       1554 allocs/op
BenchmarkNoop/noop-12                             245217              4664 ns/op           26216 B/op         22 allocs/op
BenchmarkReplace/8192-12                            5328            249861 ns/op          32.79 MB/s       34408 B/op         23 allocs/op
BenchmarkReplace/16383-12                           2366            495646 ns/op          33.06 MB/s       42600 B/op         23 allocs/op
BenchmarkReplace/32768-12                           1046           1110322 ns/op          29.51 MB/s       58985 B/op         23 allocs/op
BenchmarkReplace/empty-12                         222662              4755 ns/op           26216 B/op         22 allocs/op
BenchmarkReplace/2048-12                           17350             66603 ns/op          30.75 MB/s       28264 B/op         23 allocs/op
BenchmarkReplace/4096-12                            9249            122996 ns/op          33.30 MB/s       30312 B/op         23 allocs/op

v1.7.0:

goos: linux
goarch: amd64
pkg: github.com/extism/go-sdk
cpu: 12th Gen Intel(R) Core(TM) i7-1255U
BenchmarkInitialize/noop-12                   64          17807498 ns/op         5553731 B/op       8519 allocs/op
BenchmarkInitializeWithCache/noop-12                1300            798052 ns/op         1830382 B/op       1954 allocs/op
BenchmarkNoop/noop-12                              67366             17587 ns/op           70142 B/op         23 allocs/op
BenchmarkReplace/8192-12                            5043            247079 ns/op          33.16 MB/s       78329 B/op         24 allocs/op
BenchmarkReplace/16383-12                           2517            457056 ns/op          35.85 MB/s       86523 B/op         24 allocs/op
BenchmarkReplace/32768-12                           1292            940309 ns/op          34.85 MB/s      102912 B/op         24 allocs/op
BenchmarkReplace/empty-12                          67363             16843 ns/op           70137 B/op         23 allocs/op
BenchmarkReplace/2048-12                           15170             81431 ns/op          25.15 MB/s       72183 B/op         24 allocs/op
BenchmarkReplace/4096-12                            8074            134374 ns/op          30.48 MB/s       74230 B/op         24 allocs/op
evacchi commented 5 months ago

we released v1.7.2 a few days ago with a bunch of fixes https://github.com/tetratelabs/wazero/releases/tag/v1.7.2 main includes some improvements to the compilation times as well.

nilslice commented 5 months ago

This release may also see some improvements https://github.com/tetratelabs/wazero/releases/tag/v1.7.3

Will test it out

nilslice commented 5 months ago

Looks like we're using a now-removed API (1.7.0 -> 1.7.3):

go test -bench ./...
# github.com/extism/go-sdk [github.com/extism/go-sdk.test]
./extism_test.go:773:62: undefined: experimental.FunctionListenerFactoryKey
./extism_test.go:793:61: undefined: experimental.FunctionListenerFactoryKey
nilslice commented 5 months ago

1.3.0:

BenchmarkInitialize/noop-14                          722           1637676 ns/op         2863588 B/op       2534 allocs/op
BenchmarkInitializeWithCache/noop-14                3314            338563 ns/op         1627090 B/op       1481 allocs/op
BenchmarkNoop/noop-14                             631005              1938 ns/op           26504 B/op         22 allocs/op
BenchmarkReplace/empty-14                         583592              2049 ns/op           26504 B/op         22 allocs/op
BenchmarkReplace/2048-14                           28004             43246 ns/op          47.36 MB/s       28552 B/op         23 allocs/op
BenchmarkReplace/4096-14                           14134             83635 ns/op          48.97 MB/s       30600 B/op         23 allocs/op
BenchmarkReplace/8192-14                            7153            166118 ns/op          49.31 MB/s       34696 B/op         23 allocs/op
BenchmarkReplace/16383-14                           3462            341831 ns/op          47.93 MB/s       42888 B/op         23 allocs/op
BenchmarkReplace/32768-14                           1543            773949 ns/op          42.34 MB/s       59273 B/op         23 allocs/op

1.7.0:

BenchmarkInitialize/noop-14                          124           9525460 ns/op         6155813 B/op       8558 allocs/op
BenchmarkInitializeWithCache/noop-14                2179            487101 ns/op         1871706 B/op       1945 allocs/op
BenchmarkNoop/noop-14                             223034              5150 ns/op           70144 B/op         23 allocs/op
BenchmarkReplace/8192-14                            8060            137262 ns/op          59.68 MB/s       78343 B/op         24 allocs/op
BenchmarkReplace/16383-14                           4262            275337 ns/op          59.51 MB/s       86539 B/op         24 allocs/op
BenchmarkReplace/32768-14                           2041            587690 ns/op          55.76 MB/s      102929 B/op         24 allocs/op
BenchmarkReplace/empty-14                         182199              5555 ns/op           70140 B/op         23 allocs/op
BenchmarkReplace/2048-14                           29769             40297 ns/op          50.82 MB/s       72198 B/op         24 allocs/op
BenchmarkReplace/4096-14                           16035             72510 ns/op          56.49 MB/s       74245 B/op         24 allocs/op

1.7.3:

BenchmarkInitialize/noop-14                          158           7486412 ns/op         5319317 B/op       6911 allocs/op
BenchmarkInitializeWithCache/noop-14                2560            473021 ns/op         1851399 B/op       1806 allocs/op
BenchmarkNoop/noop-14                             141536              7226 ns/op           70139 B/op         23 allocs/op
BenchmarkReplace/16383-14                           4167            283178 ns/op          57.86 MB/s       86535 B/op         24 allocs/op
BenchmarkReplace/32768-14                           1978            604613 ns/op          54.20 MB/s      102923 B/op         24 allocs/op
BenchmarkReplace/empty-14                         161700              6700 ns/op           70134 B/op         23 allocs/op
BenchmarkReplace/2048-14                           28716             41730 ns/op          49.08 MB/s       72199 B/op         24 allocs/op
BenchmarkReplace/4096-14                           16026             74786 ns/op          54.77 MB/s       74245 B/op         24 allocs/op
BenchmarkReplace/8192-14                            7995            144315 ns/op          56.76 MB/s       78339 B/op         24 allocs/op
zshipko commented 5 months ago

There do seem to be some improvements despite there being some regressions, I think I'm in favor of updating to 1.7.3 since it probably includes some other bugfixes and improvements compared to 1.3.0

nilslice commented 5 months ago

I think I'm in favor of updating to 1.7.3

I am too! I think we just need to consider what the alternative is for the experimental Listener things are which were removed. We have tests to ensure we can still call these APIs, and I think we use them in the Observe Go SDK, so there's some additional work to figure out where they went and/or how to fix/update the code.

zshipko commented 5 months ago

It looks like we can just switch over to experimental.WithFunctionListenerFactory - I will try to update the test

nilslice commented 5 months ago

@zshipko i can't approve, given it's my PR - but if CI passes, feel free to merge! we can do some additional testing before we release.