search

bytecodealliance / wasm-micro-runtime

WebAssembly Micro Runtime (WAMR)
Apache License 2.0
4.67k stars 576 forks source link

Perhaps Performance Problems Compared With Other Runtime Tools #2544

Open luxinyi0105 opened 10 months ago

luxinyi0105 commented 10 months ago

Description

While executing the given testcase with wamr and other runtime tools, such as wasmer, wasmtime and wasmedge, wamr's execution speed is significantly slower.

Test Case

The given testcase was simply mutated a wasm file, which was obtained by compiling C program generated with Csmith using Emscripten Compiler(Emcc).

The original C program is c_file.c, the compilation results with Emscripten is wasm_file.wasm, and its wat format is wat_file.wat.

We mutated the wat file to change all i32/i64.add to i32/i64.sub, and change all i32/i64.shl to i32/i64.rotr. The result after mutation is mutated_file.wat, and its wasm format is mutated_file.wasm.

Versions and Environment

Tools Version: iwasm 1.2.3 Operating system: Ubuntu 22.04.1 Architecture: x86_64

Extra Info

Although it takes different execution times, the running results obtained by different runtime tools are the same: RuntimeError due to out of bounds memory access. The execution time of different runtime tools are as follows:

wasmer： 28.679 s wasmtime： 22.521 s wasmedge with AOT mode： 9.485 s wasm-micro-runtime： 2 m 55.342 s

I'm not sure which wamr engine is used when executing wasm files using the default command iwasm mutated_file.wasm? Is it JIT or interpreter mode?

In addition, whether the situation in this issue is related to performance problems still needs to be determined by you. Thanks a lot!

TianlongLiang commented 9 months ago

The default command chooses classic interpreter running mode, if you compile with the default CMake flag. I will check the program and see whether there is any other factor for such a performance gap for this program

TianlongLiang commented 9 months ago

When using LLVM JIT, the running time is more or less close to other runtimes:

❯ time ./iwasm --llvm-jit  09-10/mutated_file.wasm
Exception: out of bounds memory access
./iwasm --llvm-jit 09-10/mutated_file.wasm  20.22s user 0.03s system 102% cpu 19.802 total

And if you run in AOT mode:

# first compile wasm to aot file
❯ ./wamrc -o 09-10/mutated_file.aot 09-10/mutated_file.wasm
# then run aot file
❯ time ./iwasm 09-10/mutated_file.aot  
Exception: out of bounds memory access
./iwasm 09-10/mutated_file.aot  4.18s user 0.00s system 98% cpu 4.230 total

In your machine, running time maybe slightly differ, but should be close to other runtimes.

You can refer to this blog for more details on the different running modes of WAMR.

luxinyi0105 commented 9 months ago

You can refer to this blog for more details on the different running modes of WAMR.

Ok, I'll try it later. Thanks for your reply.