Add performance suite

[mindplay-dk]

I'd like to propose the additon of a basic performance suite.

While micro-performance is not my primary concern, having the ability to benchmark branches against each other could be useful when making changes or optimizations, to ensure performance doesn't degrade going forward.

I'd also be curious to include a competitive benchmark against at least one of the competing frameworks - likely the best way to do that, would be a contribution to js-framework-benchmark. I'm particularly interested in learning whether frameworks such as hyperHTML are really substantially faster because they avoid a virtual DOM intermediary. (I suspect the difference isn't at all watershed, and/or possibly using VDOM is even faster, on account of it being so much simpler overall.)

(Perhaps js-framework-benchmark could also provide the building blocks for a benchmark suite in-project?)

If you'd like me to contribute, let me know if you have a benchmark framework preference.

[jorgebucaran]

I'm particularly interested in learning whether frameworks such as hyperHTML are really substantially faster because they avoid a virtual DOM intermediary.

hyperHTML/lit-html are faster. A "more complex/scientific" patch algo could bring us very close though.

[mindplay-dk]

hyperHTML/lit-html are faster

Is that an assumption or did you benchmark something?

If pico has never been benchmarked against anything else, that's an assumption - I'd like to know ;-)

[jorgebucaran]

@mindplay-dk It is an assumption, but it's a reasonable one. There's not diffing in hyperHTML or lit-html AFAIK, whereas VDOM is all about diffing :)

I haven't benched picodom, but I did hyperapp using krausest/js-framework-benchmark.

Perf results are average compared to the other benched libraries. We are good. I do want to work on a faster diffing algo and I have something cooking, but that's just me scratching an itch. Whatever you are doing, you are fine using picodom or any other VDOM or library. Choose the one that matches your style and helps you accomplish your goal faster. To me, developer convenience > anything.

[mindplay-dk]

I just found this benchmark myself not 20 minutes ago 😆

He also posted an interactive page of results here:

http://www.stefankrause.net/js-frameworks-benchmark6/webdriver-ts-results/table.html

And man, we are more than "good" - picodom is top of the line! It's nearly on par with the very fastest libs, to the point that the difference is too marginal to care about. And it scores consistently low on all benchmarks, where a lot of libs have one or two weak spots.

My biggest surprise here is the libs that do direct-to-DOM updates in fact are not faster than picodom - some of them are actually slower and/or have 1 or 2 weak spots.

To me, developer convenience > anything.

To me, it's all about comprehension - I want to know what I'm working with, and the main selling point for me, is here's a 200-line codebase that anyone can read and understand, inside and out, no suprises, no mysterious fancy black box.

The point is picodom is fast. We should advertise that. A lot of people have heard about the new direct-to-DOM libs, and have reservations - these benchmarks clearly illustrate that VDOM doesn't have to mean anything in terms of performance, neither speed or memory usage. Picodom is awesome 😄

[maxidr]

I think that's benchmark was made using pico-dom and not picodom 🤔

[leeoniya]

not like anyone said anything 5 months ago ;)

https://www.reddit.com/r/javascript/comments/6dzd4u/build_your_own_next_generation_view_framework_1kb/di738jd/

latest results are here, btw: https://rawgit.com/krausest/js-framework-benchmark/master/webdriver-ts-results/table.html

[mindplay-dk]

@maxidr whoops, looks like you're right.

Okay, so there's no benchmark yet.

Probably the best approach would be to fork js-framework-benchmark and PR then?

[jorgebucaran]

@mindplay-dk As you wish, but results will be the same as Hyperapp's.

[mindplay-dk]

results will be the same as Hyperapp's.

@JorgeBucaran understood, though there's no saying that'll always remain the case.

Perhaps more important is to have a simple internal benchmark, to benchmark performance impact of a change.

[jorgebucaran]

...though there's no saying that'll always remain the case.

True. As both projects start to evolve away from each other, that may just be the case. Although, I hope any diff improvements to the algo also make it to Hyperapp and viceversa.

Perhaps more important is to have a simple internal benchmark, to benchmark performance impact of a change.

This could be a good start. https://github.com/hyperapp/hyperapp/pull/186

/cc @ngryman

[leeoniya]

in light of this [1], i suggest that you guys finally submit a keyed impl soon :)

[1] https://github.com/krausest/js-framework-benchmark/issues/304

[jorgebucaran]

@leeoniya Oh, I was just thinking about this the other day. It would be great if js-framework-benchmark only dealt with keyed implementations and did away with the non-keyed ones.

It would help simplify that project a bit too.

[leeoniya]

It would help simplify that project a bit too.

i dont think he's going to throw the non-keyed ones away, but probably recommend against them in the next blog post that's coming up soon (Round 7). it'd be a shame if picodom-keyed or Hyperapp-keyed was on that absentee list.

[jorgebucaran]

@leeoniya It's just a matter of adding key={item.id} to the row in the row view file. I've done it locally, but I always have to modify js-framework-benchmark source code beyond recognition so that I can get it to start. I'll get around to it eventually, not a life or death situation.

[leeoniya]

but I always have to modify js-framework-benchmark source code beyond recognition so that I can get it to start

😕

this is all i do to get it working: https://github.com/sveltejs/svelte/issues/48#issuecomment-302111192

since then, also run npm install && npm run build-prod in webdriver-ts-results. also do npm install http-server in the repo root first.

[jorgebucaran]

@leeoniya Sorry, I was not trying to criticize the project. I think we've been there at various points before hehe. My setup is more complicated and it basically takes over my machine which I also use for developing my projects as well as work.

[leeoniya]

i think you mistook a confused emoji for an angry emoji :)

i dont take anything about that bench personally; i'm just a contributor like everyone else. my goal is to get everyone in there and make sure everyone plays fair.

[jorgebucaran]

I think setting up that benchmark is not a piece of cake, but it's a superb effort by @krausest and contributors to have come up with a single solution that, despite some difficulties, can benchmark, more or less reliably, all the frameworks, including quirks and whatnot. 👏

[krausest]

@JorgeBucaran I like hyperapp and I like keyed frameworks and this is the result. (Sorry for spamming picodom)

[jorgebucaran]

I don't want to add a performance suite, but instead, use @krausest's benchmarks. I'll add Picodom there later as I am currently rewriting the diff algorithm. The diffing/patching will follow https://github.com/hyperapp/hyperapp/issues/499#issuecomment-357557785.

Sidenote: I may eventually change the name of the library to avoid the name collision issue evident in the aforementioned benchmarks. I will definitely create a separate issue for that when the time is right.