Rendering Web Components

@spiffytech and @chronoDave - thanks for keeping this alive. The two of you have the biggest say in where the project needs to head. More so than I do.

I've been away for a long while due to work-related pressure, health, and responsibilities associated with middle-age. But it looks like I have a bit of time on my hands. I am hoping to contribute something here.

While working on the performance fixes last year (which I was not able to finish), I had this idea that a lot of our complexity could go away if we embraced Web Components and rendered Custom Elements. In a way, it would be similar to Lit. But also different in an interesting way - we can keep the html markup strongly-typed instead of lit's weaker template literals.

We can probably even retain compatibility with existing code, but the rendered markup would change.

For example, we could try to get the following (example from our docs):

const HelloWorld = () => {
  return new forgo.Component({
    render() {
      return <p>Hello, world!</p>;
    },
  });
};

to render:

<hello-world>
  <p>Hello, world!</p>
<hello-world>

This would also radically simplify the architecture of forgo itself. Most of forgo is book-keeping logic, to keep Component State mounted on various DOM nodes. A large part of the complexity comes from Components not always having a node to bind to - like for example when a render() returns a list, or simply another Component instead of a DOM node, or even a null.

I could try this on a branch. What do you think?

A very early attempt here: The Web Components PR

This doesn't do everything, but the size of forgo can reduce drastically. https://github.com/forgojs/forgo/blob/53e75e84bc6a2a3c5995398453e10a6819d71423/src/forgo-next.ts

This code already renders the following:

const CounterComponent = () => {
  let counter = 0;

  return new forgo.Component({
    name: "counter-component",
    render(props, component) {
      function inc() {
        counter++;
        component.update();
      }

      return (
        <div>
          <button ref={counterButtonRef} onclick={inc}>
            INC!
          </button>
          Clicked {counter} times.
        </div>
      );
    },
  });
};

Edit: This has moved to https://github.com/webjsx/webjsx

At this point I've stepped away from Forgo. I'm rubber-stamping PRs to keep @chronoDave unblocked, but I really underestimated how much work it takes to bring a framework up to fully-featured, performant, and battle-tested, and that goalpost is moving and accelerating. I've got other projects that need me :wink:

I can still be a sounding board :blush:

FWIW I think the options available to you are even broader than adopting WCs. I think you, @chronoDave, and I are the only people who ever made serious use of Forgo, so as long as you two agree, you could do literally anything, up to and including forking or wrapping another framework, radically break compatibility, etc.

E.g., as a random idea off the top of my head that I haven't thought through at all, what about wrapping Hyperapp? I always thought it looked interesting, is small, performs well, but it could sure use a component-based API and other QoL stuff. That lets you focus on your unique value-add instead of of spending 99% of effort on implementation details that don't set Forgo apart. Idunno, just spitballing, you do you ¯\_(ツ)_/¯

Anyway, as far as switching to web components, some thoughts:

Lit templates support strong typing via TypeScript with the VS Code plugin. Frustratingly they can't mark an attribute as mandatory, due to limitations of the WebComponent API, but otherwise you do get the red squiggles:

1.1. Web components care greatly about attributes vs properties, because you can only send strings through properties. Lit says "whatever, we'll just pass that cognitive load on to the developer". I'm not sure what it means for performance, semantics, etc. if you tried to bury that by making everything a property or something.

1.2. WCs are a radical change to the developer experience of debugging, styling, etc. Also, does this play nice with e.g., Bootstrap, Bulma, Shoelace, etc.? I have no idea what Lit's story is for 3rd-party tooling that expects a specific DOM structure.

1.3. Also, Web Components require hyphenated component names. Will that be forced upon the developer? Personally I've never liked that as a mandate.

Ryan Carniato, creator of SolidJS, just yesterday had a long, long Twitter conversation about why frameworks never adopted web components, even though he personally has built full apps with them. Info to chew on.

Worth highlighting in there is that all the big frameworks (except React?) are moving away from reifying components as actual ideas at runtime. Solid and Svelte already do it, Vue Vapor is working on it, Angular too: at build-time your code is compiled into a representation where components don't actually exist anymore. The big reason here is it's a huge bottleneck on performance improvements: creating + managing all those component instances is expensive, it's very slow to walk a tree and say "something changed, let's figure out if it matters" hundreds of times. Now it's "signals", where the framework bookkeeps which DOM nodes use which pieces of data at creation time, and surgically updates only the parts of the page that use data, without any kind of diffing. Components are just syntactic sugar.

If Forgo is interested in pursuing this approach, building on S.js may be fruitful. It's actually one of the libraries that inspired Solid.

Forgo's current implementation was one of the slowest frameworks on the JS benchmarks before it was removed. Will it get faster from cutting out bookkeeping, or slower from the overhead of creating web components + their assigning every component instance a DOM element it might not need?
I'd be curious to see how much bookkeeping can really be cut. A major weakness of the browser-builtin WebComponent API is they have zero DOM/state reconciliation support. It's expected that you'll just blow away your children and recreate them, and if you don't want that... good luck? Lit gets around that by overlaying the builtin API with its own DOM diffing tech. In fact, you'll notice the Lit API looks nothing like the builtin API because the builtin API kinda sucks and pushes a very unpleasant model of components & lifecycles :melting_face:
I know a lot of Forgo's bookkeeping is related to component state reconciliation, and an awful lot of the bugs I worked on were finnicky edge cases around that. Very much not simple to get right. My concern is that this is an 80/20 problem, where it's quick to make a simple WebComponent-backed version of Forgo that works in toy scenarios, and then a constant trickle of issues found in the wild for edge cases that got missed. Porting the test suite should help a lot there — I think I wrote tests for most of what popped up.
A large part of the complexity comes from Components not always having a node to bind to - like for example when a render() returns a list, or simply another Component instead of a DOM node, or even a null.

What are you imagining for the solution here? I'd assume we don't want to introduce extra, semantically-meaningless DOM nodes everywhere. Heavyweight, messes up CSS or semantic HTML, etc. I assume you've thought of that, so I'm not clear on what you're envisioning.

Great points, and deep insights.

The problem with Lit I see is that VSCode plugin based approaches aren't anywhere as good as the TypeScript compiler based approach. In terms of integration and correctness, TS will be tested and maintained to a higher standard. I admit I haven't tried the vscode plugin; but I believe the point still stands.

1.1 Web components care greatly about attributes vs properties, because you can only send strings through properties

This is a good point. I don't know if I'm doing this correctly - I am passing all sorts of things through currently. I'll need to read up the spec in detail.

1.2 WCs are a radical change to the developer experience of debugging, styling, etc.

Agree.

1.3. Also, Web Components require hyphenated component names.

Agree.

why frameworks never adopted web components...

I've been wondering too. But otoh, a) custom elements are letting me side step (some of) the expensive runtime search which happens in forgo now to find a compatible DOM element. b) due to the way forgo works (components are linked a custom element and vice-versa), I was able to make updates very quick. I am not sure of course, there's a lot of figuring out to do and the pitfalls may not be seen early.

reactivity

I'm thinking forgo should just be a thin layer (of functions) that translates jsx transpiler output into Web Components. JSX transpiler output is a bunch of nested functions, so it'll need to be transformed into the class-based layout of Custom Elements. Everything else can be a higher-level lib.

Forgo's current implementation was one of the slowest frameworks on the JS benchmarks before it was removed.

Yes, possibly. In any case, it'd make performance work easier because 70% of the code is going to go away. Performance is my main motivation.

Porting the test suite should help a lot there — I think I wrote tests for most of what popped up.

Completely agree. I'm gonna work on some toy apps first and see where it goes.

What are you imagining for the solution here? I'd assume we don't want to introduce extra, semantically-meaningless DOM nodes everywhere.

With custom elements, all Components have a node to attach to - that is, the wrapper node. Currently, this is hugely complicated - we carry around a lot of state until we find a DOM node (for example, when "Parent2" renders "Parent1", which renders "Child1", which renders a "div"). It gets worse when there are arrays involved. With custom elements, there's no state to carry around - we can immediately bind to the custom element for the component, and each Component will have a custom element.

This doesn't seem bad to me because it's meaningful. They're things like <book-list>, <book> etc.

Next steps:

Let me get a toy app working first
Exploratory work is happening here https://github.com/webjsx/webjsx because I want to get rid of some of the other cruft as well

I have no idea if this will actually work. But let's see.

I am leaning towards making this is a separate project. I think it'll have a large delta with the current forgo, in terms of how it needs to be used and what it renders.

As someone who's really only used Forgo and not maintained it, I can only speak as an user. Whether or not moving to web components is a good idea is hard for me to say as I've got little to no understanding of the internals of Forgo. For as long as the API stays the same, I would probably be indifferent.

I have looked into using web components for small projects myself (especially because they're native) but I've always been put off by the syntax and implementation details. It seems unneccesarily complicated compared to, essentially, document.createElement().

I'm personally not too bothered about performance as I'm often not building large, complex web apps but instead simple, re-usable components or basic web apps. More often than not, the performance bottleneck is caused by myself; calling too many uneccesary rerenders.

To me, forgo fills a specific niche that React and mithril can't quite fill. React is far too large and clunky and mithril doesn't give me full control over when components get updated. hyperapp looks very interesting, but doesn't support jsx and relies on vdom which makes it hard to work with browser API's (I quite like how I can use document.querySelector or document.getElementById instead of needing ref everywhere).

Tl;dr, forgo is small, allows me to use jsx and doesn't use vdom. For as long as those remain and the API stays the same, I'm indifferent on whether or not forgo should use web components.

It will be impossible to retain the same API with Web Components, which is why I moved the code to a different repository. Like @spiffytech was saying, the underlying mechanisms are very different.

Having said that, I wanted to mention that forgo will become a lot better (2x smaller, easier to maintain, and probably faster) if we made a small change. Which is: if Components always rendered a DOM element on the outside, most of forgo's bookkeeping can be removed.

Here's an example:

This example is ideal, because we can latch on to the div (can be any other):

const HelloWorld = () => {
  return new forgo.Component({
    render() {
      return (
        <div>
          <h1>Hello</h1>
          <SomeComponent />
          <p>lorem ipsum</p>
        </div>
      );
    },
  });
};

The following is cause of complexity, because it gives us no DOM element to attach component data:

const HelloWorld = () => {
  return new forgo.Component({
    render() {
      return <SomeOtherComponent />;
    },
  });
};

If the API doesn't change, are you ok with a change like this @chronoDave?

forgojs / forgo

Rendering Web Components #107