search

alshakero / perf.zone

A Javascript benchmarking website with NPM modules support
https://perf.zone
11 stars 1 forks source link

Strange results in Chromium compared to other benchmarking sites #8

Open JobLeonard opened 5 years ago

JobLeonard commented 5 years ago

Somehow, the "switch and template string"-based snippets of this benchmark perform a lot worse on perf.zone than on the other benchmarking sites (see screenshots and links below).

I don't know if that means that there is something preventing the code from being properly optimized, or something that creates extra overhead, but it's strange. I tried measuring performance with the DevTools, but the memory/node graphs don't really clarify anything (neither memory allocation nor garbage collection seems to line up with the different sub-benchmarks in any of the tested sites).

https://jsperf.com/join-vs-template-venryx http://jsbench.github.io/#7f03c3d3fdc9ae3a399d0f2d6de3d69f http://jsben.ch/9DaxR https://run.perf.zone/view/Join-vs-Template-Venryx-1512492228976

Chromium

image image image image

JSPerf: image

jsbench.github.io: image

jsben.ch: (WTF?!) image

perf.zone: image

Firefox

This problem is not present in Firefox, but I add the results just for the sake of completion: (And yes, here jsben.ch is the benchmarking site with weird results, but that's not really a problem for perf.zone)

image image image image

Venryx commented 5 years ago

Strange indeed! (I'm the original poster of the stackoverflow question, and noticed this through a notification)

It would appear to be an issue of some benchmarking sites not being structured in a way for the JS optimizer to work properly. It makes it annoying to investigate though, because the sites all seem to have optimization "break" on different things, and in different browsers.

I concur that jsben.ch is not worth much attention though, since it doesn't actually display the operations per second. This makes it harder to compare absolute values. (eg. see whether it's due to something losing optimization, or rather the other options gaining optimization)

P.S. Dang, Firefox is fast for the switch-based snippets -- 500 million+ operations per second!

JobLeonard commented 5 years ago

P.S. Dang, Firefox is fast for the switch-based snippets -- 500 million+ operations per second!

That's usually a sign it has compiled constants away, and given that the input is a constant array that seems likely:

var parts = ["some1","some2",3,"some4",5];

If I edit the set-up a bit to randomize the input, the numbers change quite dramatically although the switches are still much, much faster:

https://jsperf.com/join-vs-template-venryx/2

I'm honestly still a bit suspicious of these numbers

Venryx commented 5 years ago

I added another test case to your set above: https://jsperf.com/join-vs-template-venryx/4

(The "Concat - switch - bracketed ("/" + part)" entry.)

Interestingly, it was the fastest of all for me in both Chrome and Firefox.