Virtual DOM for Dojo 2?

[kitsonk]

Topic

Should we consider a virtual DOM for Dojo 2?

Considerations

There are several considerations that should be made about a virtual DOM for Dojo 2:

• Perceived benefits:
  • Allowing batching/diffing of DOM operations to improve perceived user responsiveness
  • Insertion of a control point between Widgets (and other DOM operators) and the DOM
  • Ability to support serialization of DOM actions which allows for easier isomorphic code
• Perceived disadvantages:
  • Additional overhead on the client
• Should a virtual DOM include the full sub-set of DOM operations, or only those operations we would want people to use in the construct of a Dojo 2 Widget?
• Are there existing libraries that are sufficient to integrate into Dojo 2?

[pdfernhout]

A virtual DOM for Dojo2 is a fantastic idea! I feel a vdom approach obsoletes hooking up dependencies and manipulating the DOM manipulation for most webapps, including for reasons of faster development, easier testing, and avoiding memory leaks. Picking an existing vdom like Mithril, virtual-dom, or Google's incremental-dom is probably the best bet for Dojo2 to move forward quickly. Then existing Dijit widgets could be ported to that vdom system as components to ensure a11y and i18n for Dojo2 webapps.

== More details

Here are three additional benefits of a vdom approach:

• Easier and better development due to less accidental complexity. Based on my experience migrating a forty page webapp from Dijit to Mithril, a vdom approach like Mithril uses makes webapps easier to write and debug. A vdom generated from scratch from stored data is easier to reason about than a complex web of interdependent widgets and models hooked together by dependencies (especially when multiple widgets can modify the same underlying data object). I write that as someone who has been hooking up such dependencies for years using a variety of languages and GUI libraries (all the way back to Smalltalk where the idea of establishing a dependency between GUI widgets on model updates came from). Such dependencies can become problematical when two or more GUI widgets have dependencies on the same part of a model which they independently update (which creates a risk of spurious updates or even "ringing" under rare situations where GUI widgets bounce updates around between them continually). Supporting dependencies also means you can't use plain old JavaScript objects to store your data but need special models that track dependents (or your GUI needs to be more complex in other ways) which adds another layer of complexity. The vdom approach (along with a Flux-like architecture) is similar in spirit to to how most video games redraw their GUI with every frame based on the current state of the simulated world (although perhaps with clipping or other optimizations to reduce redrawing needs). For most webapps, that improved development experience will translate into more developer time spent on the essential complexity of application-specific issues instead of the accidental complexity of dependency plumbing, in the same way that automatic garbage collection is a big win for most applications in practice.
• Easier testability. Another benefit is much easier testability of GUI code. A vdom is, after all, essentially just a JSON-like object. So, it is easier to write test cases for this intermediate vdom output compared to firing up a browser/DOM emulator.
• Less memory leaks. Another benefit of a vdom approach is reducing the potential for memory leaks in long-running webapps. A vdom approach can stay closer to raw HTML because you can build a vdom by arbitrary JavaScript helper functions instead of wrapping DOM nodes with JavaScript objects. The wrapping process used by Dijit involves a registry and requires calling (or otherwise queuing) free methods at the right times to avoid leaks, which adds complexity to the GUI implementation/maintenance task. For example, I have spent many hours debugging memory leaks in nested Dijit grids, which required understanding a host of subtle interdependencies across multiple stages of the Dijit GUI construction and destruction process as they interacted with the application. That is all time that did not go into improving application functionality.

There are multiple vdom approaches out there (the main ones of which are well tested and have user communities), so creating another vdom is unlikely to be a good use of developer time for Dojo2 developers. Contributing to an existing vdom might be worthwhile as needed though. Even if Dojo2 had special vdom needs or wanted to improve integration with an existing vdom library in some specific way, forking an existing project would probably be better than starting from scratch. So the question on whether to implement a "full sub-set" is a moot issue IMHO. People are going to expect at a minimum whatever the vdom library chosen already offers, which fits with the toolkit approach.

Here is a set of recent benchmarks that include vdom libraries (although like all benchmarks they may have misleading aspects): https://auth0.com/blog/2016/01/11/updated-and-improved-more-benchmarks-virtual-dom-vs-angular-12-vs-mithril-js-vs-the-rest/#results-link

There may still be other support code that could be written for existing vdoms though. For example, I wrote a specification-driven panel builder which worked with first Dijit and then Mithril. And creating a set of consistent good-looking themeable widgets on top of any existing vdom system would be worthwhile especially as a path forward for migrating Dijit apps -- especially widgets that are i18n and a11y friendly leveraging Diijit's experience. I lost ARIA support when I went from Dijit to Mithril, sadly. Although I can add ARIA support back in at some point by more coding (and probably taking ideas from the Dijit source), that's the kind of thing a good widget set should do for me.

The HyperScript "h" function that virtual-dom uses or the Mithril "m" function equivalent is another big win for developing complex webapp GUIs, given HTML-ish templates make less and less sense these days in complex webapps. As Leo Horie, the developer of Mithril, points out:

Because Mithril acknowledges that dynamic data binding in large systems is more procedural than declarative, we can leverage the full power of Javascript without getting tripped by Greenspun's tenth rule, and without the need to learn framework-specific templating syntax that we'll probably not be using in 10 years. ... The point is that in addition to having the refactoring power of a programming language available at no extra cost, we also get access to a wide array of tools: templates can be profiled granularly, minified, linted, unit tested without the slowness of loading up PhantomJS, etc. The "Mithril ecosystem" might still be in its infancy, but the Javascript ecosystem is huge and Mithril is ready to take advantage of it in ways that many frameworks can't even begin dreaming of.

I've enjoyed working with Mithril, but that doesn't mean that Mithril is necessarily the best choice for Dojo2's default widgets. But whatever vdom makes sense (Mithril, virtual dom, Google's incremental dom, or others like cito.js), Mithril has some good ideas worth exploring including a default redraw policy for all "on" event handling callbacks and network requests. If there was a big area still in need of innovation with a vdom system like Mithril (and perhaps others), it probably involves improving the composition of more complex widgets from less complex widgets, as well as revisiting how widgets are initialized. Other changes might be required to make Mithril a bit more TypeScript-friendly (although I made a workaround which was not hard). Such changes might require forking a project like Mithril (although I am not sure). There are ongoing discussions in the Mithril community on those sorts of issues though, and none were a show-stopper for actual use.

Dojo2 could try to abstract away the differences between vdoms with a wrapping layer (as with GFX and graphics), but likely that would just make code harder to debug. A better approach towards that sort of abstraction across vdoms might be a panel builder (as I used), but that introduces its own complexities, so I'm not necessarily suggesting that either. Leveraging HyperScript to construct vdoms for any vdom library, and perhaps defining a simple set of functions to create event callbacks that either queued redraws or not might be valuable though if one wanted to support multiple vdoms. Again, it's not clear that is worth it though, as hard as it is to choose between existing vdoms given various tradeoffs.

I know of some warts with Mithril as I've used it, but other vdom libraries may have their own warts I don't know about. And Mithril gets a lot right. Still, I might expect that a pure vdom approach like virtual dom might lend itself more to the Dojo toolkit approach, where some Mithril-like qualities (but based on Dojo2 code) and some other design changes for composing complex widgets could then be built around it.

As a reminder, for some larger companies, React is a non-starter because of the Facebook patent clause in the license grant.

[dylans]

Here are a few other existing libs that are not listed in the comparison article above:

• Maqette
• diffhtml
• typescript-vdom (A one-time, apparently unmaintained port of virtual-dom to TS)
• riot
• Ractive (though it does far more than just vdom)

There was a fairly good discussion about a year ago at https://www.reddit.com/r/javascript/comments/2jav2q/is_there_any_good_standalone_implementation_of/ which included comments from the authors of Mithril and virtual-dom. There were early discussions around trying to introduce some interoperability between vdom libraries.

[pdfernhout]

Thanks for the additional examples. Ah, "The Paradox of Choice". :-) So, the biggest issue may be a choice between what is out there. Or, alternatively, deciding to punt and instead abstract the vdom choice away similar to the Dojo GFX approach (at the cost of introducing more complexity and potentially getting stuck at the least common denominator).

Key aspects of such a choice could include:

• level of commitment and friendliness of the key maintainers
• licensing (including patent license issues as with React)
• license assignment quality
• size of the community
• predicted future growth of the community (including if Dojo2 adopts it)
• number of third-party widgets
• level of testing
• level of documentation
• ease of use
• modularity, including not having extra stuff duplicating Dojo2 (like Mithril's own request function)
• being able to define widget components
• speed under varying conditions
• browser versions supported
• overall design and code quality
• likely future development directions as far as compatibility with Dojo2 priorities
• memory footprint
• experience using it
• simplicity

But even with a table of such information (some of which like documentation or code quality may be hard to evaluate from casual inspection, and others like future directions that might be mostly speculation), then you have to decide how to weigh them. And also, weight them with what sorts of projects in mind, given Dojo2 is "intended to be a highly opinionated framework, but also a toolkit"?

[pdfernhout]

As a plug for abstracting away the choice, it seems to me that this situation might be somewhat different from GFX. All of the vdom libraries target the same DOM.

The main differences a developer notices are going to be support code around that, not the ultimate functionality of what can be output.

For example these sorts of things vary across libraries:

• how are virtual dom nodes defined (the function, like h or m or whatever and the order of parameters)
• how are components defined (if they can be)
• how are components initialized
• whether templates are emphasized (like JSX with React or Google's incremental-dom)
• how are components mounted into the DOM
• how are redraws triggered (or queued or suppressed)
• how is HTML sanitized

If I think about how I spend more than 95% of my time coding a GUI with Mithril, it is writing "m' function calls and writing related event handlers.

Here is a discussion between Mithril and virtual-dom maintainers about whether the projects could collaborate in some way (but that was before Mithril added its own component system) and not that much came of it beyond perhaps some refactoring on the virtual-dom side https://github.com/Matt-Esch/virtual-dom/issues/22

Given Mithril bills itself as a framework (and as much as I like many but not all of its choices), for Dojo2, on the surface, it probably makes more sense to go with something like virtual-dom (or a similar focused library like incremental-dom) which is more modular and also has HyperScript compatibility. But I have not used virtual-dom stand-alone. Last summer, I compared Mercury and Mithril and preferred Mithril, but that was at the framework level, not in regard to just vdoms.

I'd expect the Google vdom to have the cleanest copyright history though? But I don't know how much the Google vdom is actually used withing Google or just an experiment.

[pdfernhout]

Here are some GitHub vdom project popularity stats (not that numbers are everything) in order of watchers:

Library	Watch	Star	Fork	commits	notes
typescript-vdom	1	0	0	14	in TypeScript
dom-layer	2	16	1	123
virtex	2	25	1	61
kivi	7	27	1	152
diffhtml	8	182	12	208
vidom	13	115	8	409
domvm	14	118	6	182	unopinionated, multi-paradigm
angular-virtual-dom	16	184	8	53	for Angular
snabbdom	21	372	24	151
Maquette	23	89	9	194	in TypeScript; HyperScript API; interactive tutorial; requires CLA
morphdom	24	404	18	91	not vdom but uses DOM like a vdom
Bobril	27	151	27	406	in TypeScript; inspired by Mithril and React
react-lite	30	645	28	100	drop-in React replacement
Citojs	34	255	14	68
preact	54	34	26	113	uses React API
inferno	55	582	27	2,132	differentiates static content from dynamic; uses React API
Imba	65	1,642	49	449	new programming language with vdom
deku	74	2,371	95	1,046	uses functions instead of classes with state
incremental-dom	91	1,828	65	263	from Google; reduces max memory use and GC; requires CLA
Ractive	204	4,648	318	4,294	emphasizes templates; impressive interactive tutorial system
virtual-dom	218	5,418	315	401	claims to be feature complete; stats don't include Mercury's 129 watchers; HyperScript API
mithril.js	307	5,414	413	1,337	HyperScript-like API
riot	354	8,483	730	1,998	brings custom tags to all browsers
Glimmer	1,169	15,601	3,365	11,502	stats for Ember use; vdom-ish but optimizes static areas via handlebars and streams
React	2,634	35,622	5,816	6,097	requires CLA

For reference, HyperScript has these stats (Watch 12 Star 261 Fork 2 commits 120)

[pdfernhout]

Another vdom called Bobril (inspired by both Mithril and React) is mentioned in "What is Virtual Dom". They also mentioned the W3C working draft on "Shadow DOM" as a competing idea.

vdom for Dart: https://github.com/localvoid/vdom

vdom for use with Angular: https://github.com/teropa/angular-virtual-dom

More benchmarks with several more vdoms to add to the above list: http://vdom-benchmark.github.io/vdom-benchmark/

[pdfernhout]

Bobril is in TypeScript! See bobril.ts. So are the examples.

[pdfernhout]

Morphdom may be worth looking at to see their criticism of vdom relative to their simpler approach.

Lightweight module for morphing an existing DOM node tree to match a target DOM node tree. It's fast and works with the real DOM—no virtual DOM here! ... morphdom does not rely on any virtual DOM abstractions. Because morphdom is using the real DOM, the DOM that the web browser is maintaining will always be the source of truth. Even if you have code that manually manipulates the DOM things will still work as expected. In addition, morphdom can be used with any templating language that produces an HTML string. ... The premise for using a virtual DOM is that the DOM is "slow". While there is slightly more overhead in creating actual DOM nodes instead of lightweight virtual DOM nodes, we are not seeing any noticeable slowness in our benchmarks. In addition, as web browsers get faster the DOM data structure will also likely continue to get faster so there benefits to avoiding the abstraction layer. See the Benchmarks below for a comparison of morphdom with virtual-dom.

However, without a vdom, testability may suffer. I'm not sure how they support components either. Still, its simplicity at apparently minimal performance cost has a lot going for it. :-)

[pdfernhout]

Added more updates to the table and included each project's commit count. The list is now up to twenty-two vdom-ish implementations.

[pdfernhout]

Here are some links and ideas towards supporting a suggestion that if Dojo2 is to be an opinionated framework about "data-rich web applications", generating GUIs through JavaScript/TypeScript should be made easy, and using HTML-ish templates should be an afterthought and perhaps even discouraged. This is background for another suggestion for moving Dojo2 widgets forward that I plan to make in another comment.

For balance, consider first an argument for Web Components instead of vdom and especially React: "React Is A Terrible Idea". From there:

Meanwhile, Web components are now available in Chrome and Firefox, with polyfills for older browsers. Web components are the biggest leap forward for the Open Web in years…and, yet, we're talking about React, a completely unnecessary framework. (Seriously, why not just introduce JSX and virtual DOM as separate libraries? And, as far as the data flow stuff goes, that's not even a new thing. People have been doing that for years.) This is bad, because the Open Web is important, and ten years ago, we nearly lost it. ... We have the luxury of complaining about “slow” rendering times in the DOM because we can take for granted a browser platform that is improving faster than we can even make use of the new features. This includes major rendering improvements. We don't need more workarounds; what we need is more investment in making the platform we have, whose very existence is a near-miracle, better.

BTW, here is a presentation on React weaknesses relative to virtual-dom that helped steer me away from React (use the arrow keys to navigate): "Don't React".

Answers to this StackExchange question address "Pros and Cons of Facebook's React vs. Web Components (Polymer)". The top rated answer talks about pros and cons of Domain Specific Languages (DSLs) among other issues. Templates (like in JSX or handlebars) are essentially an HTML-ish DSL for defining parts of GUIs.

This essay suggests, among other things, that Web Components could be leaf nodes of vdom trees: "What happened to Web Components?". Given that point, a vdom approach then does not exclude, say, integrating IBM's delite project at some point.

Having said that, there is a lot of current technology already for importing JavaScript libraries that define widgets into webapps. If you don't care about defining GUIs in HTML, then what are Web Components really offering? Personally, while I am all for a standard way to define complex widgets, I am not yet persuaded by Web Components since discussions of them seem to assume people will be still trying to code complex GUIs in HTML instead of JavaScript. In practice, that is just not possible. If much of the app needs to be in JavaScript/TypeScript to support complex operations to retrieve and display data, what are you getting by putting a key part of the GUI definition in some HTML-ish syntax? In general, I'd suggest the net result of trying to define GUIs that way is to make them harder to write and to refactor for programmers compared to using something like the HyperScript library. That is something I learned from using Mithril (even though Mithril uses its own HyperScript equivalent). Someone might say web designers like templates; but in practice, many good designers often can't handle any sort of coding, the templates still can't be viewed easily by designers, and it might be better to get the app running quickly so a designer could tweak a related stylesheet. At best, HTML might just serve as a throw-away mockup of what a designer might want to see. And then a tool like html2hscript could be used to convert the HTML to JavaScript calls to the HyperScript library.

For example, this essay on "A Guide to Web Components", while accurately describing the problem most webapp developers are stuck in, seems to completely miss that point that JavaScript/TypeScript can be an answer to creating complex webapps that are maintainable. From there:

Recently I was working with a client to train their internal teams on how to build web applications. During this process it occurred to me that the way we presently architect the front-end is very strange and even a bit broken. In many instances you’re either copying huge chunks of HTML out of some doc and then pasting that into your app (Bootstrap, Foundation, etc.), or you’re sprinkling the page with jQuery plugins that have to be configured using JavaScript. It puts us in the rather unfortunate position of having to choose between bloated HTML or mysterious HTML, and often we choose both. In an ideal scenario, the HTML language would be expressive enough to create complex UI widgets and also extensible so that we, the developers, could fill in any gaps with our own tags. Today, this is finally possible through a new set of standards called Web Components.

Here are are some arguments against a lot of fancy template use which presumably also apply to Web Components: "You have ruined HTML":

What’s the alternative? Simple. Do your programming in a programming language. Don’t try to embed it in some crazy will-never-be-a-standard binding expression invented by a fly-by-night JavaScript framework. You probably don’t want to go back to completely manipulating the DOM by hand, but there are options like React (without JSX, since JSX sucks for many of the same reasons above) which avoids DOM manipulation (instead favouring the idea of just describing the “current state”) and keeps all your code as code. This way, your existing tools, linters, coverage etc. will all work fine. Want to extract a function to reuse it? No problem! Want to refactor->rename? No problem (well, ish… as well as this works in JS!). Want to use TypeScript and have type-checking in your expressions? No problem (well, ish… when TypeScript+React play nicer together ;)). ... HTML is a declarative markup language. Let’s leave it as such, and stop trying to crowbar non-standard binding expressions into it.

Mithril embraces JavaScript, which is a big part of why I like it so much. Leo Horie who created Mithril makes a similar point here that I quoted from in an earlier comment. But you can do similar things with other vdoms. And if you are using JavaScript to define your GUI, if you really want to parse a template written in a DSL (in JSX or handlebars or whatever) to generate vdom nodes, you can.

[pdfernhout]

As a first strawman suggestion, Dojo2 could just use Mithril. :-)

But, here is a more complex alternative as a second proposal to make a somewhat-Mithril-like wrapper around an existing vdom library. Feel free to improve it or outline why it would not work or suggest better (perhaps more decisive) alternatives.

With the previous comment as background, here is a suggestion building on Kitson's insight that a vdom makes possible the "insertion of a control point between Widgets (and other DOM operators) and the DOM". Mithril's brilliance is in part in its "m" function, which acts like the HyperScript library to creates a vdom. However, as a "control point" in front of the DOM, that function also wraps any "on*" event handler specified with a function that by default causes a redraw after the even handling is done (I'm glossing over a few redrawing details here). Dojo2's widget system do much the same -- but of course better, learning from the Mithril community's experiences and leveraging Dojo developer's great design skill. Then Dijit2 widgets could be created using that function (whatever it is called when it is imported, "m" or probably "h", or even "d", or whatever).

For Dojo2, this vdom construction function would call an existing vdom system. It doesn't matter too much which vdom it is at the start. The virtual-dom library might be the most obvious choice as it is intended to be modular and has been around for a while and is popular, but it almost does not matter. It could be Google's as another example, but that is a bit harder to work with. It could even be Mithril.

However, that one function by itself is not enough. There would also have to be a few more support functions equivalent to what Mithril defines for mounting components into the DOM, for queuing redraws, and for some other functions. The intent of these functions would be to call the underlying vdom's methods. These functions essentially would be acting in a way similar to Dojo1's GFX library to define a stable API for vdom use. Variants of these functions could perhaps be made to support other vdoms. The functions themselves would mostly just delegate to the underlying libraries, so they probably would only be a few lines each at most. Other than the main vdom construction function, it might even be OK if they were a bit "leaky". What really matters is that 99% of the HyperScript-like code being written does not need to change if you switch underlying vdoms. Then work could commence to make Dijit2 equivalents for Dijit1 widgets including ARIA a11y support and i18n with theme support.

The trickiest part may be handling creating components. I'm doing some handwaving here in thinking the Dojo2 team will be able to come up with a good cross-vdom solution for that. I don't know if it really is possible, So, that is a weakness of this suggestion. Potentially, a builder pattern could be used instead for components if components were completely unworkable in a cross-vdom way. Or alternatively, we could just commit to one specific vdom, or perhaps just a few which had enough commonality to make possible defining components in a compatible way.

Another tricky part might be different libraries using different approaches to tagging vdom elements to improve how updates happen or to preserve DOM component internal state during complex updates like moving a lot of DOM nodes around.

One advantage of this strategy is to isolate Dojo2 to some extent from the rapidly changing world of vdoms as demonstrated by the more than twenty vdoms listed above (and no doubt there are more).

Still, the extra complexity of a wrapper could increase difficulty in debugging the code.And thinking about cross-vdom issues could be time consuming. Committing to one vdom library would certainly make things easier. But if the library was not Mithril, I'd suggest we'd still want some sort of wrapper to get the default behavior of auto redrawing after an event callback.

As a simplistic code example (and ignoring the component issue), consider a simple GUI to present a greeting and update the time when a button is clicked:

import d = require("dijit2-vdom");

var now;

function updateTime() {
    now = new Date().toISOString(); 
}

updateTime();

function render() {
  return d("div#greeting", [
    "Time: ", 
    now,
    d("hr"),
    d("span.salutation","Hello!"),
    d("button", {onclick: updateTime}, "Update time")
  ]);
}

d.mount("someExistingDivID", render);

[dylans]

Thanks @pdfernhout for providing a ton of background information. On the topic of web components, there are still some fairly major limitations with the spec. that makes us hesitant that version 1 of web components will work for us today. The approach to cross component communication, or the way style sheets get applied, and the inconsistent platform support today. Looking at the extreme effort that Polymer and Delite/Deliteful had to jump through to get something workable today gives us a significant reason to not race forward with them. Just because something is an open spec or standard doesn't mean it's the right approach.

Next, we don't really consider popularity as a criteria, other than making sure the thing we want to use is active and licensed reasonably. We need to look at how strong the internal code is, what the performance is, and what we need to do to make an API we like that wraps whatever we choose to leverage.

I will say that many of the virtual DOM APIs feel like they're going to turn code into long chains of imperative code make me think there must be a better way, as that style of coding discourages reuse.

I definitely appreciate the information and research. I just think we need to first answer the questions of what we want from our APIs for Dojo 2, before we race ahead to pick a existing solution. And a big part of that is going to be finishing our thinking about how the Dojo 2 widget system should work.

It is clear to me though, given the number of implementations already created, that we should not create yet another vdom implementation.

[pdfernhout]

I agree it makes sense to reflect on APIs for Dojo2 and to think more about Dojo2 widgets before picking supporting libraries.

That said, I still feel it is almost certain that an imperative HyperScript-like API is both a good choice for a modular interface to whatever vdom implementation is underneath that HyperScript layer and a good choice to support whatever Dojo2/Dijit2 abstractions is above that HyperScript layer. That is based on my enjoyment in using such a HyperScript API via Mithril, and a desire to help other developers have similar fun too. :-)

I'll create a separate issue for discussing the pros and cons of a HyperScript-like API for Dojo2/Dijit2.

[pdfernhout]

@dylans I'd agree popularity of a vdom is not the main consideration -- otherwise I would not have chosen Mithril compared to, say, React. Popularity is just one factor among many as a reflection of likely community support (something easily skewed like by Facebook's association with React, or Google's with incremental-dom). And such tables are just a point in time. To the extent popularity matters in any way, ideally, I'd like to see popularity trend chart of each library of interest on GitHub. I'd also like to see a commit trend chart -- but even then, as with virtual-dom, a "feature complete" library may stop having many commits, so that does not necessarily mean anything bad.

And with the internet, it's also amazing how popularity can seemingly change overnight when something goes "viral". Google's incremental-dom, for example, not doubt started out small in followers last summer, and is rapidly picking up interest given the big name connection plus addressing an important concern other vdoms have not focused on. I expect incremental-dom is going to continue on that upward trend for at least a while (until other vdoms begin to copy that idea).

BTW, in reviewing that chart again (and checking if Mithril really had a suspicious-looking 1337 commits), I realized that I had put in the wrong popularity stats for Mithril. It has more than 10X more watchers than I first listed. I think what happened is that I got confused between a development "mithril" project as well as a mainline "mithril.js" project, and I pulled the Watcher stats from the development project but the commits from the mainline one. Those numbers in the table are all now from the mainline project -- although I left the commits at "leet" instead of the current 1118, since I like that sentiment for Mithril and @lhorie. :-) And so Mithril now edges out virtual-dom in popularity in the chart. I have to admit, I was happy to move Mithril forward quite a bit in the list. :-)

To be fair, I've also realized that were I to combine Mercury's Watcher stats with virtual-dom, as the Mercury framework uses it, virtual-dom would come out slightly ahead of Mithril again in popularity. Possibly there may be other popular framework projects that use virtual-dom too. So, that sort of situation is another confusion in trying to make sense of popularity figures given project interdependencies.

The next time I make such a comparison table, I probably should write a webapp to pull together all the current stats rather than doing it by hand (assuming such a tool does not exist already). Perhaps such a GitHub project comparison table would make an interesting future Dojo2 widget demo? But maybe we don't want to encourage people to look at popularity first by making it too easy to compare, leading to a "rich get richer" runaway first mover advantage for poorly written projects? That's a perennial problem with "like" buttons and such.

Even with that risk though, a table sorted by current popularity at least provides some context as to what is going on out there. I learned a lot putting it together starting from your suggestions of other vdoms to look at; thanks!

I also learned from that exercise that someone could easily spend many days or even weeks trying out these vdoms, evaluating their code and documentation, and seeing which ones were better in some way. I spent days just deciding between React, Mithril, and Mercury/virtual-dom. I remain happy enough with my choice of Mithril, so for me personally, Dojo2 aside which may other priorities, I feel no great urge to switch from Mithri. I say that even as I feel the component aspect of Mithril is a bit awkward, although not unworkable.

[pdfernhout]

@dylans On whether Dojo2 will eventually have its own vdom, when I knew of about four or five vdom libraries in the table, I was feeling, why reinvent the wheel? But having found out about twenty-three vdoms and counting, I'm thinking, maybe there is some reason for making more vdoms related to optimization for specific use cases, now that the idea is getting widespread? Either that, or writing vdoms is an interesting, fun, and not-too-hard exercise? :-)

React/Facebook patent licensing issues aside, the reason there are so many vdoms seems to be that people are experimenting with various approaches that emphasize different things. It's apparently been worthwhile for the JavaScript community for some people to do such experiments in general, to test new ideas, since several new ideas are emerging from such experiments.

So, while there are other things that interest me more personally, it is not inconceivable to me that other people involved with Dojo2 might want to try their hand at vdom creation at some point either for some specific optimization purpose or even just for fun and self-education. There is probably also nothing like trying to write a vdom to make someone appreciate the ins-and-out and tradeoffs of existing vdoms as a learning experience. Obviously, whether such an effort should be a priority relative to other needs is a different matter, as well as whether it would be a better learning experience to just tinker with an existing vdom first. I learned a lot about Mithril's vdom trying to debug an issue I was seeing (which turned out not to be a bug so much as an undocumented feature related to not setting initial data on an input field). So, contributing bug fixes and documentation and such to an existing vdom is another way to learn a lot about them too.

[pdfernhout]

One way to move forward on the Dojo2 vdom decision at some point in the near future could be for @kfranqueiro and me to work together doing some pair programming (or whatever) to create a proof of concept of a demanding dgrid dataset running under the Mithril vdom. That does not have to commit Dojo2 to Mithril, nor would it need to involve a substantial port of dgrid. The point is just to reduce the risk of Dojo2 adopting a vdom because supporting dynamic grids well under a vdom is likely to be the most significant challenge of the effort. With a convincing proof of concept, the Dojo2 project would be in better position to commit resources to moving forward with a vdom. That said, even if the proof of concept fails or is unconvincing, Dojo2 can still use the existing dgrid with a vdom system like Mithril. I've done that before to integrate with D3 graphs into a Mithril app. It adds a bit of interfacing complexity do things that way rather than a pure vdom-only solution, but that sort of integration is doable and worthwhile when the value of using the library on its own terms outweighs the small special integration cost (as is certainly the case with dgrid). So, demoing dgrid-as-is running under Mithril alongside some vdom widgets could be another part of such a proof-of-concept as a first step, and then afterwards, we could see what we could do with a vdom-powered grid displaying a large amount of data.

== More details

Performance (in terms of responsiveness, CPU use, memory use, and random GC pauses) seems to be a big driver motivating the creation of so many different vdom libraries as experiments (especially out of performance concerns for mobile and low-end devices). A modern smartphone in a materially prosperous country typically now has several fast CPU cores (MediaTek is planning one with ten cores to out-compete Qualcomm), gigabytes of memory, and a big battery that is usually having most of its energy wasted anyway with the phone being woken up every few seconds to ping the network by badly written apps (and malware). In a few years, most people who want them will have similar phones. In that context, a part of a webapp where, say, a user enters an address into a form or scrolls through a short list is unlikely to need a lot of optimization for CPU, memory, or battery use.

So, within a broad range of tasks, it seems performance of a straight-forward vdom approach (like virtual-dom or Mithril) is good enough for a broad range of UI use cases. That does not mean optimization is not still important, but it means optimization probably can be a lower priority relative to functionality and maintainability in most typical UI cases. And other libraries optimizing for performance exist already like vidom, inferno, and incremental-dom, so likely things will only get better as far as performance.

The place where performance matters the most in terms of application area, and where vdoms (or at least, vdom-powered applications) tend to have noticeable performance issues is with large scrolling grids of complex widgets. A similar demanding task would be supporting cell-dependent mouse hovers when moving the mouse quickly across a big table. Such tasks are likely to be common though in Dojo2, given the emphasis on "data-rich web applications". So, ultimately, a Dojo2 version of dgrid is what will probably determine the best choice for vdom.

That said, this Mithril article by @lhorie "An Exercise in Awesomeness" links to a JSFiddle with a scrolling list of 5000 items using occlusion culling that scrolls quickly. I've tried it with 50,000 items and it still scrolls quickly. As another example, mithril-infinite is a Mithril library intended to support infinite scrolling. Example for that are here and here (although the table example of mithril-infinite seems to have a redraw bug on my Chromebook when I scroll a lot).

So, given the awesomeness of dgrid itself, I feel it probably could be adapted to run well on almost any vdom. Still I'm not 100% certain of that given how dgrid likely relies on receiving updates about specific changes. Would dgrid need to work differently if such updates were not available and it just knew that the data might (or might not) have changed? Perhaps the dgrid API might need to change in relation to applications signaling dirty ranges of data changes for performance reasons, if dgrid cached a copy of the vdom nodes it generated to do minimal updates on them?

Performance differences among vdoms in the range of, say, 20% overall probably will not be as big a consideration as things like maintainability, ease of use, or similar factors. Performance differences in the range of 10X in important areas would be a more determining factor. Looking at the benchmarks document previously linked above, all the vdoms there (incremental-dom, virtual-dom, Mithril, citojs) seem within about 10% of each other as regards frames per second (although the article suggests the subjective feel for incremental-dom does not match the graph).

Still, any optimization of the underlying vdom is likely to pay off, since if 1000 webapps are all 20% faster, that is still a big win (if it does not come at a huge cost in other areas like maintainability). While there are a variety of performance innovations in the above list of over twenty vdom libraries, there are two innovations that stand out to me as upcoming optimizations that may become widespread. One is Google's incremental-dom which reduced memory use and garbage collection pauses. The other is vdoms that focus on immutability of some sort to reduce diffing effort (either by using immutable vdom nodes or by marking nodes as being static based on the originating templates). So, it seems to me that eventually Dojo2 will end up using a vdom that has both of those characteristics.

Having tuned GC performance before under Java (where a full GC of a multi-gigabyte long running app under some GC configurations could take ten seconds and cause timeout errors), I can appreciate the potential benefit in responsiveness to Google's incremental-dom approach by reducing GC load through producing less ephemeral objects and less big obejcts. Most typical benchmarks will probably not show the implications of GC issues as the GC may only happen long after the benchmark test is over. Also, I'm not sure how far JavaScript VMs will be tuned in the future to use improved GC algorithms. In any case, reducing maximum memory use is a good thing too, which incremental-dom also does.

As an aside, it seems to me that, building on the immutable/static idea, it might be computationally and GC efficient for something like dgrid to have a vdom structure where each vdom node had an associated revision number (perhaps from a timestamp) that indicated its most recent mutation. As with git, you would update the revision numbers all the way down the vdom tree to the root node if you modified a leaf node in the vdom (while other unaltered leaf nodes and branches would retain the same revision number). Then the diffing algorithm could know that no DOM updates were needed for parts of the vdom tree that had the same revision number as a previously rendered section. That may be already how the static template optimization works; I'm not sure. Normally, for most GUI parts like form data entry or small lists, you might not set the revision numbers as the overhead and implementation cost might outweigh any likely performance benefits and the diff algorithm would do an exhaustive comparison of data. But for something like dgrid (with potentially thousands of components displayed on screen even with occlusion culling), you probably would set those revision numbers. It might be worth evaluating existing vdoms to see if they support such a feature or something equivalent via "immutable" JavaScript objects. Still, using such an approach would likely mean tying dgrid tightly to a specific vdom implementation (well, at least without some additional cleverness to abstract away vdom structure manipulation). Or, ultimately, such needs might drive Dojo2 to innovate with its own vdom tuned towards displaying massive amounts of data.

In any case, the need to make dgrid work even better may be the use case most likely to eventually drive Dojo2 towards having its own vdom, or at least, to switch vdoms as more perfomant ones were developed by other projects. So, I'd suggest @kfranqueiro (the main committer for dgrid now) ideally should be involved in any vdom choice, because a future version of dgrid stands to be affected the most by that choice. And given all his excellent work on optimizing dgrid, where it would not surprise me if dgrid may have helped inspire React, if Dojo2 was to have its own vdom eventually, he'd be the obvious person to create and maintain such a vdom system if he wanted.

I'm confident all the other typical Dijit-like widgets will perform reasonably well with a vdom. If there is a stumbling point in vdom adoption, it will be in relation to big grids. So, it might make sense to take some complex and demanding existing dgrid dataset and make it work with a vdom. Porting all of dgrid itself is not required for such an evaluation -- just enough to show a proof of concept on any vdom. If that example can be made to work in a satisfying enough way, then Dojo2 could move forward with confidence to a vdom. While I doubt he needs my help to do this, I'd be happy to work with @kfranqueiro towards such a proof of concept in Mithril (just as the vdom I currently know best -- a different one like virtual-dom would be OK too).

As mentioned at the beginning, even if the proof of concept was not successful, the existing dgrid could still be used in a primarily vdom-ish webapp by integrating it as a component like via Mithril's config attribute. So, since that integration seems relatively easy, it might make sense to start with demoing that integration. Then afterwards we could see whether a roughly similar grid (as far as scale of content) could be made to work under Mithril to address any performance concerns about display updating or mouse interaction.

[dylans]

@kfranqueiro has been working on defining what dgrid 2.0 needs to be, and he's certainly interested in the DOM aspects surrounding it.

I don't think we're ready to start a POC for dgrid 2 based on anything yet, as we're still planning out the APIs.

There are many ways to evaluate various vdom approaches, but my point was that, other than determining if a project is dead, popularity won't really inform our decision. It's a balance of how well the existing API will work within the approach we take for Dojo 2. Ideally we would find an implementation that is:

• Fast and performant
• Has code that is understandable, clean, and well-documented
• Has an API (or ease of wrapping the API) that will feel consistent with the other parts of Dojo 2
• Is authored in TypeScript, or at least has a community that is interested in keeping robust typings up to date
• Has an open contribution and governance model (CLAs, process to become a committer, reasonable BSD/MIT compatible license, etc.)

I've asked a friend who has a startup to measure JS code quality to do a quick comparison of all of the vdom implementations listed above, to see if that tells us anything interesting.

Again, trying to keep this short as I think we'll have time to discuss this in more depth soon, but I'm short on time today.

[pdfernhout]

For balance, an article by someone at Google against React vdom on mobile due to performance reasons: https://aerotwist.com/blog/react-plus-performance-equals-what/

A response by the React team: https://discuss.reactjs.org/t/reactjs-perf-on-mobile/781

Note also that various vdoms claim to be much faster than React.

[pdfernhout]

@dylans I'll be curious to see the results of the code quality analysis.

On "ideally" and CLAs -- based on looking for "CONTRIBUTION" files at the top level of each vdom's GitHub repo, and then reviewing those files, CLAs are apparently only currently required by Maquette, Google's incremental-dom, and React. So, all the others are non-starters unless they could be persuaded to start using them and get previous committers to sign up. So, if a CLA is essential and the other projects won't add one, then putting an API in front of Maquette or incremental-dom might be the only two feasible possibilities (assuming React is disqualified for license reasons).

Maquette is appealing to me as it both is in TypeScript and has a HyperScript API, but I have not tried it. Putting HyperScript in front of virtual-dom is likely doable though.

[pdfernhout]

I went through the online Maquette tutorial which is written using Maquette itself. It is pretty good overview, although maybe a little too hard in a couple of spots for an intro tutorial if you have not used the HyperScript API before. Based on that and looking at some of the documentation, Maquette (not to be confused with the Dojo Foundation's Maqetta) seems like it might be an OK fit for Dojo2. Maquette claims to be "very UNopinionated by design", requires a CLA (I don't know about other governance aspects), and would not overlap other Dojo2 functionality. It is designed to be fast (but I have not seen benchmarks) -- although it achieves that by pushing a few probably minor design constraints back to the programmer. The core library itself is currently 1158 lines of TypeScript and at first glance the code quality looks OK. So, subject to further code review, and the outcome of discussion about HyperScript pros and cons in issue #16, and exploring the governance model more, Maquette probably satisfies all the criterion @dylans highlighted in his last comment.

Based on my very limited understanding of Maquette's capabilities from spending a little time with the tutorial and looking at the docs, here are some things that seem to be missing in Maquette relative to Mithril and which would likely need to be added to Maquette eventually:

• the ability to cancel scheduling a redraw in an event handler, like when you don't process a keypress
• improvements for components to support code called when a widget is discarded from the vdom
• improvements for components to maybe pass in constructor functions instead of pre-made components (but maybe not since Maquette has explicitly decided not to do that and provided a work around which might be easier to understand and perform better than how Mithril does it).
• integration of DOM-based third-party components (e.g. D3 graphics) like Mithril can do via a "config" attribute, like to, say, integrate dgrid-as-is into Maquette -- but maybe it is possible already somehow?
• supporting multiple "mounted" components within one HTML page -- although maybe it can do that already if you make multiple "projectors"? [Update: Maquette supports any number of render functions per "projector", each associated with a different DOM node. But, you can't selectively run only some render functions in a projector (they all run in "doRender"). That might be a performance issue in some extreme cases if you knew for sure you only needed to rerender one component on screen (say, a status indicator) and not the rest (say, a big grid). I'm still not sure if you can have more than one projector in a webapp.]

However, for most of these (except changes to component construction which might not be needed anyway), if there is not a way to do it already in Maquette, I'd expect the Maquette project's core maintainer (@johan-gorter) might welcome a pull request as long as it does not affect performance much.

Other aspects of the Mithril framework like startComputation/endComputation probably could be handled (perhaps better) with Dojo2 Deferred and an explicit redraw. (One benefit of using Deferred is that you would not break the redraw system if you forget to call endComputation like in an error handler as can happen with Mithril.) The absences of Mithril's route and request support in Maquette could be seen as a feature given Dojo2 would have its own versions.

Based on a very limited perusal of the Maquette code, the code seems a bit clearer to me than Mithril's. That may be because it does less though? Being in TypeScript helps make it more readable, too.

So, subject to further testing and discussion, Maquette seems to me like a good-enough candidate for a Dojo2 vdom. One way to learn more about it would be to make a simple project in it using Dojo2 -- especially one tested on a mobile device.

[johan-gorter]

Hello @pdfernhout,

Thank you for writing such an extensive review about maquette. I can see you really dug into it. Let met clarify some of the points you brought up.

• First of all, maquette claims to be UNopionated. Consider it to be a library/utility instead of a framework. This means that it can easily be embedded into an opinionated framework such as Dojo.
• Maquette provides the afterCreate and afterUpdate callbacks which are meant for integrating third-party components. This should be flexible enough to integrate both DOM-based and virtual-DOM-based widgets.
• A lot of points you mention are about performance. In my experience large pages on small devices always render and diff within 16ms, so any optimization is unneeded. But just to reassure you, maquette also provides a caching mechanism (createCache) which allows you to bypass rendering and diffing of entire subtrees. It is also possible to have multiple projectors on one page.
• The CLA is only there to protect maquette from people who contribute patented code. We hope that GitHub makes CLA's unneccessary in the future.
• Maquette has taken a different approach to components than other frameworks. Creating and deleting components/widgets is not done implicitly during rendering, but it is done explicitly. This makes maquette perform better and makes it simpler.

I think the last point is something that you will want to investigate. What I understand from Dojo/dijit is that widgets are created explicitly and put onto a form/layout. Imho this makes maquette a better fit than other frameworks that create and destroy components on the fly.

One last remark: maquette will also announce support for transpiling from JSX soon.

[dylans]

Thanks for taking the time to provide an informative response @johan-gorter.

To clarify on the point of CLAs, we actually think that requiring one is a good thing! With our work through the Dojo Foundation and now the combined jQuery Foundation, we think they encourage people to only contribute code when they have the right to contribute code.

And yes, with Dojo 1.x, we are fairly explicit about the lifecycle of a widget, and I believe we would want to do the same for 2.0.

It sounds like maquette could be a good fit for Dojo 2. We'll let you know if we have any questions or issues once we get a bit further along.

[pdfernhout]

@johan-gorter Thanks for the reply. Sorry about the typo in my review which I have fixed; I had meant to write "UNopinionated", which of course makes Maquette a great choice for integration into other frameworks.

On Maquetts vs. Mithril components: I agree Maquette's component approach looks more straightforward than Mithril's recommended approach, even as it may expect the programmer to sometimes do a bit more work to get the benefit of more clarity and greater performance. Having implemented and used several Mithril components, I'm still a bit confused about exactly how Mithril components are initialized and with what data and when. :-) And I'm apparently not the only one with such a confusion, as there are Mithril issues on bugs/features related to that. Mithril got a lot of stuff right, but components in Mithril have always felt a bit problematical to me because they seem to trade simplicity for ease of use in some specific situations. You provided an innovative example for Maquette on widget array handling with a helper function, which is the most difficult case to manage for components used with a HyperScript API; thanks! :-) When using Mithril, I ended up using a similar component approach to Maquette's in cases where I was creating a larger widget from other subwidgets (not an array though) where I would initialize a subwidget that was implemented as a TypeScript class, store a reference to the widget myself in the larger widget, and then just call the subwidget's "calculateView" render method myself in a larger widget's render function passed to Mithril. Maquette's Do-It-Yourself (DIY) component initialization approach makes that the expected behavior definitely seems like a better fit for Dojo2 then Mithril's in that sense -- and being simpler, is at less risk of doing unexpected or inconsistent things.

On the difficulty of writing good APIs: As @dylans pointer out in an insightful comment on issue #15: "Simple + simple + simple === simple, whereas easy + easy + easy === complex." To be fair to Mithril though, which I do value for ease-of-use among other things including a friendly user community and the choice of a HyperScript API, @lhorie was pioneering in a lot of these areas with Mithril as an elegant vdom-based framework that was one of the first React alternatives. So, it is easy to point out some issue like component initialization complexity that may only be obvious with 20/20 hindsight -- but which may be difficult to fix without a breaking change to address emerging concerns only now obvious in retrospect. The whole field of software development is littered with APIs with warts on them for that reason. It is just difficult to write great library APIs when you have to consider so many sometimes conflicting issues (including ease of use and modularity) and yet also want to respect early adopters by avoiding breaking changes. Pushing those design choices into other modules can make a lot of sense if you can think of a way to do that though modular design -- although at some point, in order to ship stuff, someone still has to make an opinionated choice even if just in what modules to combine into a framework or application.

On the difficulty of making choices about how modular to be: Modularity can have its costs like pointed out in this excellent Mithril issue discussion on modularity pros and cons between Leo Horie and @raynos (who created the extremely modular Mercury framework which uses virtual-dom). That conversation mentions Dojo modularity as an example in passing, BTW. That was a discussion that encouraged me to try Mithril over other choices (among other considerations). Of course, there are good arguments for Mercury's more modular approach too, and indeed I did not use Mithril's router as I has my own (starting from Dojo's routing approach), and I did not use Mithril's request method either (as again, I had my own already). Mithril has worked well for me even as I did not use a few parts of the framework -- and even if a more modular focused vdom library like Maquette might be a better fit for Dojo2. Still, you don't have to use Mithril's component approach to use the Mithril vdom, since you could use a strategy similar to Maquette's with Mithril, as I did sometimes with Mithril as mentioned above. Of course, if you are not going to use Mithril's component approach, or its router, or it request function, and so on, then that part of the Mithril codebase is just extra code to add unneeded complexity possibly leading to bugs and developer confusion. Modularity remains under discussion for Mithril, however, and may be part of a rewrite. Ideally, we could all focus more effort into fewer and more performant vdom modules, and then let frameworks put opinions on top of them. But that is easier said than done given some assumptions (like redraw strategies or component initialization) that get woven into vdoms, which is why it might be hard for Mithril to just start using, say, Maquette for its vdom (at least, probably without changes on both sides). And of course, there needs to be room for experimentation to make the next best thing to some purpose -- so as hard as the choice may be between vdoms, it is good there are choices.

On third-party components and Maquette: Thanks also for the pointers to afterCreate/afterUpdate to address third-party component integration. I see references in the Maquette source and a couple of examples, like the use of afterCreate to do a focus in the TODO example and tests. Is there a Maquette example somewhere already of a DOM-based widget being created (even just a div with text in it made by DOM methods) and then doing some cleanup action (even just logging to the console) when the widget was removed from the vdom (perhaps even just via "onunload")?

On performance optimization for vdom: There are apparently a couple of other vdom issues to consider beyond refresh rate (especially for mobile use) related to other aspects of performance. One issue is memory use, both the maximum memory used and also the garbage collection (GC) demand due to ephemeral objects, because GC both introduces random latency into an application and also uses energy from CPU use. Google's incremental-dom attempts to address those two memory-related concerns (maximum and GC use) -- but at possibly a cost in other areas like perhaps complexity in interfacing to that library? The other issue is reducing computation in general reduces energy use, which on mobile increases battery life. I would expect Maqette is already doing pretty well on doing the minimal amount of computation needed for the vdom task, which would be reflected in its speed. In practice I don't know how much any of those other optimizations (beyond what Maquette does already) are significant in practice or limiting factors. I'm just pointing them out as potential benefits for optimization even when frame rate is already as good as it can get.

[pdfernhout]

Tangential discussion on the Mithril mailing list: "No more MVC" https://groups.google.com/forum/#!topic/mithriljs/j48dTQvV-qA

Leo Horie (Mithril author) comments on the "No More MVC" article here (as Mithril calls itself an MVC framework): https://news.ycombinator.com/item?id=11105193

Yet another vdom (domvm) author mentions his project in response (a project inspired in part by Mithril but focused more on vdom than being a framework): https://github.com/leeoniya/domvm

The domvm author writes: "I must say, I've also become disillusioned with MVC for front-end frameworks, actually after using Mithril [1]. I'm sure you've seen me lurking and asking questions in the repo since before Mithril got big. I used it on a few projects and decided the controller was not an abstraction that made much sense. I looked elsewhere and stumbled upon domchanger [2] but after trying to use it seriously and forking some changes also ran into many impasses. This is what inspired me to develop domvm [3], a simple, pure-js composable vdom view layer for plain models. I took a few of Mithril's good ideas, removed globals, made the magic as opt-in modules (so a bit more wiring is needed) and ended up with something extremely fast (at least 2.5x Mithril), composable and mixed imperative/declarative. It's been a good journey and I'm very happy where everything ended up. Thanks for some inspiration :)"

I'm not sure yet how domvm really differs from Maquette or virtualdom. It claims its (HyperScript-ish looking) templares are a superset of JsonML: http://www.jsonml.org/

Leo Horie praises aspect of domvm and says he is considering some ideas from it for a new Mithril rewrite. He then goes on to say: "It goes both ways. I never expected that Mithril would inspire new projects, and I've come to really appreciate the great insights that come from these projects. At the end of day, we all want better software, and having more hands on deck (even if each person is doing their own thing) is a net positive in my opinion. JS fatigue critics can say all they want, but this is how innovation happens :)"

Leo Horie also says there (which is a good summary of three ways of building JavaScript GUIs, Dijit1 being most similar in approach of these three to Knockout): "The pain point that templating engines address is automating the process of figuring out what DOM changes are caused by what state changes. In order to do that, you have to either dirty check the state tree (as Angular 1 does), dirty check the template tree (as React/Mithril/vdom does), or have an observable state tree (as Knockout does). If the templating engine defers its responsibility to the developer, then if, for example, you do a dataList.pop(), you are responsible for writing out the code to remove the last item in a DOMNodeList, or updating some count label, or whatever else the view may be doing. This works ok in a small app, but it tends to become hard to maintain as a codebase grows in size (due to requirement changes or whatever)."

Another link mentioned on the Mithril mailing list which relates to the supporting MVC infrastruture (and of course mentions Mithril positively): https://github.com/ciscoheat/mithril-hx/wiki/Rediscovering-MVC

This post is not to promote Mithril specifically. I can see how a more modular vdom library integrated into Dojo2 is likely a better choice than Mithril (although Mithril may be more modular soon). This is to point out ongoing discussion and the issues innovators in the vdom field are currently concerned about.

[pdfernhout]

As a datapoint, I created some experimental code that integrated Dojo2 (loader and core, used to make a network request) with Maquette and three.js, and it all seemed to work OK. The advice by @johan-gorter to use afterCreate worked perfectly for integrating a div managed by three.js.

[leeoniya]

domvm author here.

@pdfernhout

I'm not sure yet how domvm really differs from Maquette or virtualdom.

virtual-dom is a raw vdom-diff lib, so it's not quite in the same category as React, Mithril, domvm, etc. It is also incredibly slow, large and offers very few features, there is little reason to consider it for anything these days. If you're looking for high perf and prefer to roll your own components, i would instead recommend kivi, snabbdom or citojs. If you're looking for a few more features, Preact and Inferno are very fast replacements for React. domvm is slightly slower than kivi, cito and Inferno, and faster than Preact and Mithril by a large margin and snabbdom by a smaller margin.

I'm not too familiar with Maquette but from browsing the site and the code, it is strongly focused on animation chaining and has no benchmarks to evaluate its perf. It also has no router (dunno if you're needing one). It is likely you'll run into many things that are not batteries-included in it.

What I wanted to do with domvm is get Mithril's features and much better perf without strict structural enforcement or odd "controller" concepts and globals-everywhere. domvm is decoupled, so the modules are independent of each other - you can simply use view without the others:

• view (vdom/diff)
• route (router)
• watch (mutation observers)
• html (isomorphism)

View is about ~10k min. Everything together is ~15k min and ~6k gzipped.

I made a quick demo a while back of implementing an adapter over domvm that would allow some existing Mithril code to work unmodified. It's a tiny thing that shims some Mithril-isms, like the structure, controller, global observers/auto-redraw and arg-passing: https://github.com/leeoniya/domvm/tree/master/demos/mithril-adapter. You can see the link rotator demo from Mithril's home page copy-pasted: http://leeoniya.github.io/domvm/demos/mithril-adapter/

If you guys have any questions about the lib, feel free to ping me.

cheers! Leon

[pdfernhout]

I've been working on a series of demos using Maquette and Dojo2 that increase in abstraction from HTML to plain HyperScript to a specification-driven system. They are available here: https://github.com/pdfernhout/dojo2-maquette-demos

[pdfernhout]

@leeoniya I haven't looked much at domvm; thanks for the information.

A router is not important for Dojo2 as it has its own -- it's actually a negative in a way as more clutter. I'll have to learn more about the value of "watch". Being in TypeScript would be a big plus for a vdom for use with Dojo2 -- although domvm looks to be in JavaScript. Thanks also for the great perspective on the entire field of vdoms.

I agree Mithril's controllers and component approach ultimately feel like a negative because of the added complexity. In the code mentioned above I have some component examples using Maquette's (simple) approach and, with a helper function or two, they seem reasonable so far.

I expect the core of Dojo2 would be usable with multiple vdoms. It would be great though if more vdoms could agree on common APIs to make it easier for common utilities and components to work with them -- but that may be too much to expect with so much innovation going on right now. But some are close with very similar HyperScript support.

[leeoniya]

@pdfernhout

I'll have to learn more about the value of "watch"

Its only value is to reduce manual .redraw() invocation & boilerplate in ajax/fetch, event handlers, and and model props' changes.

I expect the core of Dojo2 would be usable with multiple vdoms. It would be great though if more vdoms could agree on common APIs to make it easier for common utilities and components to work with them -- but that may be too much to expect with so much innovation going on right now. But some are close with very similar HyperScript support.

I think it will be a while before a consensus is reached. As you said, there is a lot going on in the vdom space. However, the performance race (kicked off by Ember's Glimmer engine) is starting to finally level off. Certainly, hyper-script is catching on, but there is an incredible amount of compositional and component ergonomics beyond the skin-deep template syntax. While Mithril, domvm, snabbdom, and others look the same from glancing at the templates, their ergonomics and limitations are vastly different. Because of this, I don't know how much you can rely on "pluggable" vdom impls because even the internal diffing strategies have performance implications and subtle intricacies which can make them incompatible. The raw vdom libs are the lowest common denominator where a consensus may be reached perhaps but it leaves you to code your own opinions over them anyhow.

We'll see :)

[pdfernhout]

@leeoniya I added domvm to the list above, to bring the total to 25 vdoms (along with adding Imba, which I also just learned about too).

I agree with you that design choices within vdoms are going to affect how pluggable they are. Maquette, for example, has "three rules" that have to be followed on how the HyperScript API is used, so it would be easier to go from Maquette to a different vdom without those rules than vice-versa. Mithril, as a framework, has its own request system that triggers redraws, so code doing network requests moved from that framework to another would have to be changed. Also, any assumptions about what the internal vdom structure looks like (whether the vdom is stored as objects, arrays, or even functions) is going to vary across vdoms. That makes code harder to port if it uses "transformer functions" as in this Mithril blog post on "When CSS lets you down". And as you point out, there may be performance issues that affect application design choices in other ways. Even the basics of setting up the association between a vdom and a DOM node and calling redraw are non-standard across all vdoms.

That said, it is going to be a lot easier to move between any vdoms (even between the React API and the HyperScript API) than it would be to move between, say, Angular 1.0 and Ember. Personally, I feel the HyperScript API is a significant innovation -- even with open-ended issues about how that API is used (including how components work with it) which, as you say, we will see about going forward.

I also like the approach taken by Mithril (and also now some other vdoms) of assuming the GUI needs to be redrawn when the user does something with it. That feels more maintainable to me than the React approach with setState (or even Flux/Reflux/Redux) where redraws are still closely tied to monitoring specific changes in data. I feel the React design (as a first generation vdom) is still conceptually hanging on to the "observable" data idea for supposed efficiency which adds unneeded complexity. Mithril as a second generation vdom improved on that by assuming any user action (or network request) makes the data dirty. Now we are seeing third-generation vdoms and frameworks like Maquette or domvm that emphasize modularity, performance, or other features -- building on lessons learned from the first and second generation vdoms and their surrounding frameworks.

Dojo1 provided common abstractions above different browser DOMs. I can wonder if it would make sense for a project like Dojo2 to provide common abstractions above vdoms? It might be good for Dojo2 or another project to catalog the most common "opinions" or design patterns about how to build above low level vdoms and perhaps provide support functions that work across multiple vdoms. Such may be in progress somewhere already?

Individual vdom frameworks are often making specific implementation choices for specific opinions. A specific choice of implementation based on an opinion may be best for performance and maintainability compared to the cost of an abstraction layer. But all those specific different opinions committed to code leaves an application programmer with a difficult choice about what vdom opinions to commit to for their own application when facing a list of twenty-five vdoms.

That situation is a bit like in the early days of the web when people either just coded websites for a specific version of the then-popular Internet Explorer (React these days?) or wrote ad hoc tests and workarounds for different browser vendors and versions (often painfully with unexpected gotchas and lots of duplication of effort across many developers). Dojo1 emerged as a way to address that siuation, as did other toolkits and frameworks (including eventually jQuery). The situation is not quite the same though since the vdoms above are all FOSS and so can be understood more easily than proprietary browsers. Also, there are also far fewer total differences between vdoms than differences across the browser landscape -- although some of the vdom framework differences may be bigger compared to many small browser quirks?

While I applaud all the innovation across so many vdoms, figuring out what to do about this proliferation of vdoms is a challenge when you have an application to write. :-)

[pdfernhout]

Building on my previous comment on similarities to the previous browser war situation, because you can bundle the vdom with your application, the vdom situation is also fundamentally different than the browser war situation. The issue here with a vdom abstraction layer is more managing technical risk from a specific vdom choice you make (as an application developer). The vdom you picked might not be maintained in the future and there might be a future security issue or browser compatibility issue. You might want to use some third-party component written for some other vdom. Or there could be some other reason for your switching between vdoms like licensing, performance, testability, integrating non-vdom libraries, deduplication when integrating two apps using different vdoms, or even just familiarity with different vdoms as developer teams change. The value of an abstraction layer is to help manage those risks, both for an application developer and also for a component writer (as a component written to some abstraction layer might in theory work on more vdoms).

Of course, new "standard" abstraction layer runs the risk of just adding to the confusion in the absence of vdom authors collaborating to some extent to support it: https://xkcd.com/927/

I expect there could be a common abstraction layer between any vdoms using the approach Mithril or Maquette takes (GUI actions make the data dirty and trigger a redraw) without too much difficulty. I'm not so sure that abstraction layer would integrate well with the React approach though (where changing component internal state triggers a redraw from that component on down). Those are two very different redraw architectures -- although a Flux/Reflux/Redux approach might be a middle ground? But even providing an abstraction layer just across the Mithril-like vdoms might be a big step forward.

[kitsonk]

I think we have had enough feedback on the virutal DOM for the time being.

Thanks everyone.

Going to lock the topic, so we can focus on something else.

[kitsonk]

I want to thank everyone for their thoughts and comments.

We have now done sufficient prototyping and research with MaquetteJS to feel comfortable that having a virtual DOM as part of Dojo 2 is likely to provide a largely performant way of interacting with the DOM. Also, we feel comfortable that HyperScript is a comfortable way for us to express DOM structures (versus something like JSX).

There are clearly a plethora of options for efficient DOM interaction currently, including the good work by Google with incremental-dom which sort of reverses the vdom paradigm.

While concepts like Web Components try to unify the pillars of web technologies present in the browser, upon lots of contemplation and experimentation, we feel that abstracting the "logic" from the rest of the technology is a clearer paradigm for us. We believe this promotes:

• Code isolation
• Code testability
• Less fragility when standards change. The more coupled the whole unit is to the 3 distinct APIs of CSS, DOM and ECMAScript, the more challenging it becomes to maintain those if such things are dispersed throughout your code.

Taking those things into account, it is then logical to take the path of vdom to abstract interactions with the DOM. Even then there are several options for virtual DOMs these days, but there were several aspects of MaquetteJS we liked:

• Built on TypeScript, which gives us a better level of integration but also brings the benefits we see of TypeScript to the vdom implementation
• The very specific focus on just being a virtual DOM. We feel that many of the other vdoms don't have that "be really good at just one thing".
• Straight forward and easy to understand API. We feel the API is logical and have discovered it is straight forward to build additional functionality on top of it. We also noted that MaquetteJS has followed the factory model versus a constructor function/class model, which is a similiar decision we have settled on in Dojo 2.

While it is not possible at this point to dictate that all DOM interactions for Dojo 2 will utilise a virtual DOM, we believe at this point a virtual DOM will be the base for Dojo 2 widgets.

So, I am going to consider this issue closed.