Performance Issue related to Cranelift

[hungryzzz]

Summary

Hi, I run the attached case in different Wasm runtimes(after being compiled by Emscripten), and I also find some performance differences between wasmer and other 3 runtimes: the execution time(collected by perf-tool, probe begins when starting to execute the wasm code(inner_module_run in wasmer) and end in sched:sched_process_exit) in wasmer is 3.5x slower than which in wasmtime.

• wasmer: 2271270.05 us
• wasmtime: 610519.54 us
• wasmedge (AOT): 430803.42 us
• wamr (AOT): 418358.5 us

Hardware & OS

• Ubuntu 20.04
• CPU: Intel(R) Core(TM) i5-9500T CPU @ 2.20GHz
• Memory: 32GB

Emscripten

• emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.24 (68a9f990429e0bcfb63b1cde68bad792554350a5) clang version 16.0.0 (https://github.com/llvm/llvm-project 277c382760bf9575cfa2eac73d5ad1db91466d3f) Target: wasm32-unknown-emscripten Thread model: posix

Wasm runtime version

• wasmer: wasmer 3.2.0
• wasmtime: wasmtime-cli 8.0.0
• wasmedge: build from commit 381b7b28049b968297e6a585b92d1cba955def66
• wamr: iwasm 1.1.2

Additional details

I find that both wasmer and wasmtime use the cranelift as default compiler. I guess maybe it's related to the different version of cranelift. And then I try to use LLVM as the compiler (wasmer run --llvm), I get the execution time 423130.01us, which show more relation to the current cranelift in wasmer. So is it convenient to upgrade the version of current cranelift? Or how can I do it?

flops-4.txt

[syrusakbary]

There's indeed something very wrong here. We need to look into this. @ptitSeb this might be the reasoning behind the JavascriptCore difference?

[ptitSeb]

@hungryzzz I build a wasm version from the source using zig to target wasi (binary attached)

I use a simple time command and I do see a difference when I compare wasmtime 8.0 vs current wasmer version using Cranelift compiler. I will check to see if it's a missing optimisation that provoque that (or maybe add optimisation option to the cli). flops.zip

[ptitSeb]

So, I have done some more analysis and comparison. Wasmer for now use Cranelift codegen v0.91.1, wich is the version you can find in Wasmtime v4.0.1, while Wasmtime v8.0 use Cranelift v0.95 If I use Wasmtime v4.0.1, I see similar result between Wasmer and Wasmtime (can you confirm this on your side?) So it's just that newer Cranelift have improved it's efficiancy. We will update Cranelift version, but not sure when.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

Feel free to reopen the issue if it has been closed by mistake.