knqyf263 / go-plugin

Go Plugin System over WebAssembly
MIT License
586 stars 30 forks source link

Add performance benchmarks #47

Closed dmvolod closed 1 year ago

dmvolod commented 1 year ago

Issue #, if available:

Description of changes: Added performance benchmarks iniside fields and host functions tests

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

codefromthecrypt commented 1 year ago

merged in #48 I think

codefromthecrypt commented 1 year ago

here's me using these to test before/after wazero update

# from before 1.0.2 commit
$ go test -run='^$' -bench '^Benchmark.*' ./tests/host-functions ./tests/fields -count 6 > old.txt
# from 1.0.2 commit
$ go test -run='^$' -bench '^Benchmark.*' ./tests/host-functions ./tests/fields -count 6 > new.txt
benchstat old.txt new.txt
goos: darwin
goarch: arm64
pkg: github.com/knqyf263/go-plugin/tests/fields
          │   old.txt   │           new.txt            │
          │   sec/op    │   sec/op     vs base         │
Fields-12   11.20µ ± 2%   11.11µ ± 3%  ~ (p=0.818 n=6)

          │   old.txt    │             new.txt             │
          │     B/op     │     B/op      vs base           │
Fields-12   1.398Ki ± 0%   1.398Ki ± 0%  ~ (p=1.000 n=6) ¹
¹ all samples are equal

          │  old.txt   │            new.txt            │
          │ allocs/op  │ allocs/op   vs base           │
Fields-12   24.00 ± 0%   24.00 ± 0%  ~ (p=1.000 n=6) ¹
¹ all samples are equal

pkg: github.com/knqyf263/go-plugin/tests/host-functions
                 │   old.txt   │           new.txt            │
                 │   sec/op    │   sec/op     vs base         │
HostFunctions-12   8.551µ ± 4%   8.774µ ± 3%  ~ (p=0.240 n=6)

                 │   old.txt    │              new.txt               │
                 │     B/op     │     B/op      vs base              │
HostFunctions-12   5.696Ki ± 0%   5.763Ki ± 0%  +1.17% (p=0.002 n=6)

                 │  old.txt   │            new.txt            │
                 │ allocs/op  │ allocs/op   vs base           │
HostFunctions-12   25.00 ± 0%   25.00 ± 0%  ~ (p=1.000 n=6) ¹
¹ all samples are equal

looks like we are using a little more memory for the HostFunctions bench in 1.0.2. Not something we need to hunt down, but anyway glad we can see things like this now! good job

dmvolod commented 1 year ago

merged in #48 I think

Yes, sure. Thanks for review and merge. Will close this PR.