Updates benchmarks to reify DOM.

[slightlyoff]

The benchmark results for this library currently rely on an apples-to-oranges comparison. This patch updates the tests to cause each library to actually create DOM.

[coveralls]

Coverage remained the same at 90.879% when pulling a5740ce53036d2944848c5b792267ea262e08310 on slightlyoff:benchmark-fix into f11287c3daf6fb8b65888e5581c4bd22e0dab2b8 on marko-js:master.

[mlrawlings]

I've already replied on twitter but I'll copy my string of tweets here to maintain a record in this issue.

TL;DR - The benchmark is measuring creation of nodes to be used in diffing. Not the diffing/patching/actualizing process.

Actualize isn't called on purpose.

////////////////

VDOM, as you point out, is about convenience. Although it's not actually VDOM that's convenient, but rather diffing/patching.

////////////////

Older versions of Marko did diffing/patching without VDOM. They essentially used innerHTML to create a node & diff with document.

////////////////

Except in cases where the tree was extremely different, this node never was inserted into the document. It was only used to compare.

////////////////

We wanted to compare switching from this approach to a VDOM. Hence the benchmark to measure creating the tree to use in diffing.

////////////////

The README for the project clearly describes this:

Creation time - the time it takes to construct a [virtual] DOM tree

////////////////

No actualize call needed. And, if you look, there's no call to ReactDOM.render. React is not actualizing nodes in the benchmark either.

////////////////

So yes,

"What we've learned here today is that creating JS objects w/o DOM backing is cheaper than creating DOM."

///////////

We knew it would be faster, the question was how much?

////////////////

We wanted to see if VDOM performance gains were enough to justify not using innerHTML. And, with up to 30x improvement, we decided it was.

////////////////

It's a micro-benchmark, for sure, but it helped us make a decision (and justify to our team/managers).

[patrick-steele-idem]

This benchmark was intentionally designed so that we could independently measure the cost of constructing a [V]DOM tree and walking a [V]DOM tree. It also allowed us to compare the cost of using arrays for storing child nodes versus using linked list (we found linked lists to have much better performance). Also, when you look at real world characteristics of an app I bet you will find that during updates that only a few nodes are added, removed or updated after the initial render. Therefore, the cost of actualizing a DOM node is not going to have a big impact on most web apps.

This benchmark is pretty old and the source code in this repo is no longer used by marko anymore (or any other project that I am aware of). We ended up merging in the VDOM implementation that Marko uses into the main marko repo. I will add a disclaimer to the README that is is no longer maintained or used, but I don't think there is any need to make changes to this repo.

If you want to use this code as a basis to set up a separate benchmark that measures reify/actualizing costs please feel free.

[slightlyoff]

At a minimum, the Benchmark Results page also needs to be updated to clarify that the section labeled DOM creation does not, in fact, create or measure the cost of any DOM construction. Similarly, the DOM Tree Walking section is mislabeled.

Better yet, a long-form explainer (such as what was provided above) would make a good preamble for the results.