search

bytecodealliance / sightglass

A benchmark suite and tool to compare different implementations of the same primitives.
Apache License 2.0
69 stars 33 forks source link

Add a pure Wasm image classification benchmark #270

Closed adambratschikaye closed 4 months ago

adambratschikaye commented 4 months ago

Add a benchmark which performs image classification in pure Wasm using tract and ONNX. This can be useful to track the performance of float heavy computations.

adambratschikaye commented 4 months ago

I think the rebuild failure is due to libsodium now requiring zig 0.12, but the Dockerfile uses zig 0.10.1.

adambratschikaye commented 4 months ago

Rebuild failure should be fixed in https://github.com/bytecodealliance/sightglass/pull/271

jlb6740 commented 4 months ago

@adambratschikaye I was not familiar with tract but was introduced by your issue here: https://github.com/bytecodealliance/wasmtime/pull/8313, so thanks for submitting. I think getting this in now is great but ultimately maybe this directory could be refactored to include multiple inferencing scenarios.

adambratschikaye commented 4 months ago

I'm not exactly sure what's up with the current failure. It looks like the mac tests are failing with:

stderr=```" INFO  sightglass_cli::benchmark > Using benchmark engine:
/Users/runner/work/sightglass/sightglass/engines/wasmtime/libengine.dylib
Error: dlopen(/Users/runner/work/sightglass/sightglass/engines/wasmtime/libengine.dylib, 0x0005): tried: \'/Users/runner/work/sightglass/sightglass/engines/wasmtime/libengine.dylib\' (mach-o file, but is an incompatible architecture (have \'x86_64\', need \'arm64e\' or \'arm64\')),
\'/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/sightglass/sightglass/engines/wasmtime/libengine.dylib\' (no such file),
\'/Users/runner/work/sightglass/sightglass/engines/wasmtime/libengine.dylib\' (mach-o file, but is an incompatible architecture (have \'x86_64\', need \'arm64e\' or \'arm64\'))
Error: benchmark subprocess did not exit successfully\n"```

so maybe something is up with the cached wasmtime engine for mac?

jlb6740 commented 4 months ago

@adambratschikaye Looks like you were right. Not sure the corruption, but it was using an incorrect cached version of the wasmtime-engine. I removed the cached image for macos and reran so that it had to build again, and now all tests pass.