[benchmark] WASM data for WebML workloads

intel / webml-polyfill

Deprecated, the Web Neural Network Polyfill project has been moved to https://github.com/webmachinelearning/webnn-polyfill

Apache License 2.0

161 stars 46 forks source link

[benchmark] WASM data for WebML workloads #184

Closed ibelem closed 5 years ago

ibelem commented 5 years ago

Liftoff is enabled by default in V8 v6.9 (Chrome 69), and can be controlled explicitly with the --liftoff/--no-liftoff and chrome://flags/#enable-webassembly-baseline flags in each, respectively.

Liftoff: the new baseline compiler for WebAssembly in v8. https://v8project.blogspot.com/2018/08/liftoff.html

We need to understand the performance improvement for ML use cases as WASM is used in our polyfill, WebDNN and OpenCV.js for ML workload. Collect latest data is necessary.

ibelem commented 5 years ago

@Christywl the stable 69.0.3497.81 (Official Build) has been released, please start the testing when you have time and add comparison report.

Christywl commented 5 years ago

Benchmark data with Chrome stable on Mac and Android: Test URL: https://brucedai.github.io/nt/benchmark/index.html Iterations: 1(warming up) + 200

MacBook Pro(10.13.6):

Model	Backend	68.0.3440.106	69.0.3497.91
MobileNet	WASM	503.96+-6.87	486.94+-4.11
SqueezeNet	WASM	298.77+-5.56	284.27+-3.46

Google Pixel 2(8.1.0):

Model	Backend	68.0.3440.91	69.0.3497.91
MobileNet	WASM	1731.64+-11.20	1570.34+-6.96
SqueezeNet	WASM	1017.54+-7.42	919.45+-7.09

huningxin commented 5 years ago

Thanks for the results! As liftoff improves the code generation, it would be good to measure the improvement of warming up time as well. @BruceDai

BruceDai commented 5 years ago

From collecting data with counting warming up time in, this results are similar as above. Collected data, so close it.