[Sticky & fixed] It looks the performance logging options affect the performance of frameworks.

[ismail-codar]

I'm working on a new framework called fidan. The actual result on the right according to the screenshot below. Create rows benchmark is slow.

When I investigated the cause of the slowdown I saw that it was related to https://github.com/krausest/js-framework-benchmark/blob/master/webdriver-ts/src/forkedBenchmarkRunner.ts#L342. If I change this line totraceCategories: "devtools.timeline,blink.user_timing", I see the result on the right. Pretty fast.

I don't know why, but there seems to be something wrong.

Additional note: When I logging timings to the console from http://127.0.0.1:8080/frameworks/keyed/fidan, I see fast results similar to those on the left.

[krausest]

That difference looks worrying!

As a first quick check I ran a single create rows benchmark for a few frameworks. I got the following results:

Could indeed be that the old code is slightly slower than with your change (but I need more measurements to make sure this isn't due to some background process and last published results are also a bit faster), but it's nowhere close to the difference you observe. What machine and platform are you running on? Is the difference for other frameworks similar large on your machine? Could you please post a PR for fidan such that I can take a look at the difference on my machine? (I would be fine with a unpolished, temporary version)

[ismail-codar]

For example, I saw that vanillajs was no different. But there is a big difference in my framework. My framework is functional reactive style so maybe related to it or something else. I will prepare PR. Thanks for the your great working.

[ismail-codar]

Hi @krausest I just sent https://github.com/krausest/js-framework-benchmark/pull/551 PR. Please note: Running it from http://localhost:8080/frameworks/keyed/fidan/ looks fast (console logging) similiar to changing this line to traceCategories: "devtools.timeline,blink.user_timing"

[krausest]

I got the PR from İsmail and ran the create 1,000 rows benchmark for all frameworks. Here's the comparison: Upper screenshot is results table from lighthouse's traceCategories, lower from İsmail's traceCategories:

Frameworks with a notable difference are:

• Marionette
• Svelte
• Fidan (not in the screenshot for lighthouse's traceCategories it's about 260 msecs vs. 180 msecs)

I'm not sure about Datum. Numbers differ, but not as much as for the others and std deviation was very high for the lighthouse traceCategories.

lighthouse's traceCategories are: -*, disabled-by-default-lighthouse, v8, v8.execute, blink.user_timing, blink.console, devtools.timeline, disabled-by-default-devtools.timeline, disabled-by-default-devtools.screenshot, disabled-by-default-devtools.timeline.stack İsmail's traceCategories are: devtools.timeline,blink.user_timing

@paulfalgout @Rich-Harris @ismail-codar Sorry - seems like my selection of traceCategories penalised your frameworks. I'll update results soon.

Any idea what's causing this issue?

[paulfalgout]

Marionette by itself looks the same, it's the jquery bundle that's so different. So something with jquery I guess ¯_(ツ)_/¯ Doesn't seem like jquery 3.4.0 addresses anything like that, so it might just be the way things are on that one.

[krausest]

Updates results have been posted (all framework for create row. marionette, svelte, datum, rx-domh for all benchmarks) . Thanks a lot for reporting this issue! I'll leave it open for further comments and for others to notice!

[krausest]

It really seems as if svelte was badly penalised. New results for the keyed version (svelte was between datum and binding.scala in the old version):