Summoning the holy 2-phase commit renderer

This is more a less a scratchpad of random thoughts and experiments related to that so far. I thought it'd be cool to do a bit more work in the open :tada:

Motivation

We've been talking a lot internally about possible directions we could explore when it comes to our renderer. One of those paths that has come up from time to time is to split rendering into 2 distinct phases: Diff and Commit.

Currently our renderer applies the changes as they are diffed. Make no mistake, this has worked really great for us so far! With concepts like Suspense and Progressive Hydration there is a need to skip or outright discard work-in-progress trees.

For Suspense the use case is rendering fallback content like a loader when a component down below notifies the Suspense component that it needs to wait on something (maybe a network request). In that case the Suspense component can decide to render a spinner or something similar. To prevent showing the user a half completed tree before discarding it, we'd need to lazily apply the changes to the DOM. This is commonly referred to as tearing.

Progress Hydration is different in that it allows the renderer to bypass certain sub-trees. Think of example like a sidebar that becomes only interactive once the user hovers it. For that to work we'd need a different piece to our puzzle that allows us to drag the existing DOM nodes around. Right now we always discard them, because they don't match the internal shape described in the vnode.

Experiments

The past two days I did play around with the idea a bit to get to learn more about the problem space and edge cases (tbh I was a little bored from working on devtools all the time...).

Splitting up `diffProps` into `diffProps` + `commitProps`

This is fairly easy as it's a rather isolated change in our renderer. I made a branch 2-phase-commit-experiment where diffProps is split into two functions. The approach in that branch is to store the work in a sort-of queue directly on the vnode.

const vnode = {
  type: "div",
  // ...snip
  _updateQueue: ["class", "foobar", "onClick", () => null]
};

The shape of _updateQueue is basically [name, value, name, value,...]. I haven't checked other approaches so far, but I think we could easily swap it for something else. Extracting this part was pretty straight forward.

Splitting up `diffChildren`

Our current renderer relies a bit too much on the DOM. We've spoken about it from time to time, that we wish to base more on the vnodes themselves. In the long term my gut feeling is that we want to get rid of .nextSibling as much as possible.

If we do go forward with a 2-phase renderer we'd have to calculate the ordering of children as before, but would need to store them somewhere. Remember that changes will only applied after the whole tree/sub-tree has completed the diffing phase. The natural solution to store those indices is on the vnode directly.

But more about that later.

The mighty `diff()`

Ohh lord, this one is tricky! The thing that makes it a bit weird is that some aspects of the implementation are a bit stuck in the 8.x mindset. It returns a DOM pointer and uses that to append it to the parent if needed.

Same is true for Fragments which have an additional _lastDomChild pointer. Back then it was the right decision, but with a 2-phase renderer we could simplify this a lot. I'd even go so far as to say that we don't need them for Fragments anymore. We could just bypass them during iteration.

One interesting detail is, that we'd still need to pass a DOM pointer around through diff() because we need it to grab the DOM attributes off of it during our first render.

Render Queue

The more I think about it, the more I wonder if we should change the way we queue work. Currently we have a flat array that's sorted by the depth of the vnodes in the tree. From my current understanding this wouldn't really work anymore with a 2-phase renderer.

Because we need to store the work somewhere, we're a lot more constrained. If we store the work on the vnode we'd need to do a second traversal in the second phase over the current tree/subtree. If we do store them in a separate data structure we'd need to do a bit of bookkeeping as to which unit of work belongs to which vnode. There is probably more to it, but I still need to wrap my mind around all the intrinsic details.

These are my findings so far, let me know what you think :+1:

preactjs / preact