krausest / js-framework-benchmark

A comparison of the performance of a few popular javascript frameworks
https://krausest.github.io/js-framework-benchmark/
Apache License 2.0
6.87k stars 842 forks source link

What changed in the WASM memory usage measurements at Chrome 104? #1218

Closed gbj closed 1 year ago

gbj commented 1 year ago

If you compare benchmark runs for Chrome prior to 104 and Chrome 104 and later, you see a sudden increase in memory usage across every benchmark for every WASM framework, even without a version bump in the framework. In most cases, this leads to a reversal of the results when comparing similar JS and WASM frameworks.

Chrome 103 OSX: Sycamore uses less memory than Solid, Yew uses less memory than React

Screenshot 2023-03-20 at 11 33 01 AM

Chrome 104 OSX: Sycamore uses more memory than Solid, Yew uses more memory than React

Screenshot 2023-03-20 at 11 33 26 AM

Granted Sycamore had a version bump in there, but Yew and React are identical versions.

I'm curious if you know what happened in the way memory use is measured or maybe in the way Chrome reports it, or possibly even in the way Chrome allocates it.

krausest commented 1 year ago

Between Chrome 103 and 104 we added a memory metric for 10k rows. And there was a discussion that lead to a different method to compute the memory usage (see https://github.com/krausest/js-framework-benchmark/issues/1050#issuecomment-1172715942 ). We're now using performance.measureUserAgentSpecificMemory() and thus the results changed.

gbj commented 1 year ago

Thanks. That at least answers the question of why it changed. It's not clear to me that it's a particularly useful measurement in its current form, given the counter-intuitive and systematic WASM vs JS discrepancy, but I'll yield to your judgment and the judgment of the folks involved in the discussion who know much more about Chrome performance stats than I do!