tc39 / proposal-signals

A proposal to add signals to JavaScript.
MIT License
3.24k stars 58 forks source link

Do the benefits of signals as a language feature outweigh the costs of solidification into a standard? #220

Open devmachiine opened 3 months ago

devmachiine commented 3 months ago

I think its great that there is a drive towards standardization of signals, but that it is too specialized to be standardized here

Q: Why are Signals being proposed in TC39 rather than DOM, given that most applications of it are web-based?

A: Some coauthors of this proposal are interested in non-web UI environments as a goal, but these days, either venue may be suitable for that, as web APIs are being more frequently implemented outside the web. Ultimately, Signals don't need to depend on any DOM APIs, so either way works. If someone has a strong reason for this group to switch, please let us know in an issue. For now, all contributors have signed the TC39 intellectual property agreements, and the plan is to present this to TC39.

(please let us know in an issue - that's what this issue is for)

Signals is a interesting way to approach state management, but it is much more complicated of a concept compared to something like a PriorityQueue/Heap/BST etc, which I think would be more generally useful as part of javascript itself.

What problem domains besides some UI frameworks would benefit out of it? Are there examples of signals as part of a language feature in other programming languages ?

What would be the benefit of having signals baked-in as a language feature over having a library for doing it?

When something is part of a standard, it's more work & time involved to make changes/additions, than if it was a stand-alone library. For signals to be a part of javascript, I think there would have to be a big advantage over a library.

I can imagine some pros being

Benefits being part of javascript, the same being true if it is part of the DOM api instead

I can imagine some cons being

Q: Isn't it a little soon to be standardizing something related to Signals, when they just started to be the hot new thing in 2022? Shouldn't we give them more time to evolve and stabilize?

A: The current state of Signals in web frameworks is the result of more than 10 years of continuous development. As investment steps up, as it has in recent years, almost all of the web frameworks are approaching a very similar core model of Signals. This proposal is the result of a shared design exercise between a large number of current leaders in web frameworks, and it will not be pushed forward to standardization without the validation of that group of domain experts in various contexts.

I think in the very least a prerequisite for this as a language feature should be that almost all of the web frameworks use a shared library for their core model of Signals, then it would be proven that there is a use-case for signals as a standard, and much easier to use that shared library as an API reference for a standard implementation.

If anyone could please elaborate more on why signals should be a language feature instead of a library, this issue could serve as a reference for motivation to include it in javascript. :upside_down_face:

mlanza commented 3 months ago

Although it may have moved away from the term, Elm as a language had signals baked in from its inception.

Clojure, rather than signals, adopted CSP (core/async). So, yes, some languages have attempted to standardize message relay.

But, like you, I share some apprehension about someone codifying their best idea of what makes a good signal library. RxJS and Bacon are popular but different. And RxJS is quite large. From my own experience, I have gotten by with signals with just a few primitives. I have not needed the sophisticated variations provided by RxJS.

I am smitten with signal goodness, but I'm content to lean on standalone libraries (namely, my own), such is the variety in the implementations.

dead-claudia commented 3 months ago
  • Increased performance
    • Higher than an order of magnitude?
    • How much CPU does signal-related processing consume on a website?
  • Easier for a few web frameworks and "Some coauthors" to use signals

Benefits being part of javascript, the same being true if it is part of the DOM api instead

  • Less code to ship
  • Better debugging posibilities

I can imagine some cons being

  • Not being generally useful for the vast majority of javascript programmers
  • Changes and improvements take longer to implement across all browsers
  • Not being used after a decade, when we discover other approaches to reactive state mangement
Q: Isn't it a little soon to be standardizing something related to Signals, when they just started to be the hot new thing in 2022? Shouldn't we give them more time to evolve and stabilize?

A: The current state of Signals in web frameworks is the result of more than 10 years of continuous development. As investment steps up, as it has in recent years, almost all of the web frameworks are approaching a very similar core model of Signals. This proposal is the result of a shared design exercise between a large number of current leaders in web frameworks, and it will not be pushed forward to standardization without the validation of that group of domain experts in various contexts.

I think in the very least a prerequisite for this as a language feature should be that almost all of the web frameworks use a shared library for their core model of Signals, then it would be proven that there is a use-case for signals as a standard, and much easier to use that shared library as an API reference for a standard implementation.

If anyone could please elaborate more on why signals should be a language feature instead of a library, this issue could serve as a reference for motivation to include it in javascript. :upside_down_face:

@devmachiine @mlanza You might be surprised to hear that there is a lot of precedent elsewhere, even including the name "signal" (almost always as an analogy to physical electrical signals, if such a physical signal isn't being processed directly). I detailed an extremely incomplete list in https://github.com/tc39/proposal-signals/issues/222#issuecomment-2119369345, in an issue where I'm suggesting a small variation of the usual low-level idiom for signal/intereupt change detection.

It's slow making its way to the Web, but if you squint a bit, this is only a minor variation of a model of reactivity informally used before even transistors were invented in 1926. Hardware description languages are likewise necessarily based on a similar model.

And this kind of logic paradigm is everywhere in control-oriented applications, from robotics to space satellites. And almost out of necessity.

Here's a simple behavioral model of an 8-bit single-operand adder-subtractor in Verilog, to show the similarity to hardware. And yes, this is fully synthesizable. ```verilog // Adder-subtractor with 4 opcodes: // 0 = no-op (no request) // 1 = stored <= in // 2 = out <= stored + in // 3 = out <= stored - in module adder_subtractor(clk, rst, op, in, out); input clk, rst; input[1:0] op; input[7:0] in; output reg[7:0] out; reg[7:0] stored = 0; always @ (posedge clk) begin if (rst) stored <= 0; else case (op) 2'b00 : begin // do nothing end 2'b01 : begin stored <= in; end 2'b10 : begin out <= stored + in; end 2'b01 : begin out <= stored - in; end endcase end endmodule ``` Here's an idiomatic translation to JS signals, using methods instead of opcodes: ```js class AdderSubtractor { #stored = new Signal.State(0) reset() { this.#stored.set(0) } nop() {} set(value) { this.#stored.set(value & 0xFF) } add(value) { return (this.stored.get() + value) & 0xFF } subtract(value) { return (this.stored.get() - value) & 0xFF } } ```

To allow for external monitoring in physical circuits, you'll need two pins:

Then, consuming circuits can detect the output's rising edge and handle it accordingly.

This idiom is very common in hardware and embedded. And these aren't always one-to-one connections. Here's a few big ones that come to mind: - Reset buttons are normally connected to several circuits in parallel. And these are often wired up to both main power (with an in-circuit delay for stability) and dedicated reset buttons, making it a many-to-many connection. When this rises, the circuit is reset to some default state. - SMBus has a one-way host clock wire, but also a two-way data wire that both a host and connected devices can drive. This two-way data wire could be thought of as a many-to-many connection as sometimes (though rarely) you even see such busses have more than one host on them, complete with the ability to drive them. And the spec does provide for a protocol for one host to take over for another. - SMBus interrupt signals (SMBALERT#) are normally joined together (a "wired OR" connection in electronics jargon) and connected to an alert input/GPIO pin in a host MCU. This lets connected sensors tell the host to re-poll them while ensuring the host continues to retain full control over the bus's clock. (This side channel is needed to avoid the risk of high-address devices becoming unable to notify a host due to losing arbitration to frequent host or low-address talk.) It's not as common as you might think inside a single integrated circuit, though, since you can usually achieve what you want through simple boolean logic and (optionally) an internal clock output pin. It's between circuits where it's most useful.

Haskell's had similar for well over a decade as well, though (as a pure functional language) it obviously did signal composition differently: https://wiki.haskell.org/Functional_Reactive_Programming

And keyboard/etc events are much easier to manage performantly in interactive OpenGL/WebGL-based stuff like simple games if you convert keyboard events to persistent boolean "is pressed" states, save mouse position updates to dedicated fields to then handle deltas next frame, and so on. In fact, this is a very common way to manage game state, and the popularity of just rendering every frame like this is also why Dear Imgui is so popular in native code-based games. For similar reasons, that library also has some traction in highly interactive, frequently-updating native apps that are still ultimately window- or box-based (like most web apps).

If anything, the bigger question is why it took so long for front end JS to realize how to tweak this very mature signal/interrupt-based paradigm to suit their needs for more traditional web apps.

dead-claudia commented 3 months ago

As for other questions/concerns:

I think in the very least a prerequisite for this as a language feature should be that almost all of the web frameworks use a shared library [...]

A single shared library isn't on its own a reason to do that. And sometimes, that library idiom isn't even the right way to go.

Sometimes, it is truly one library, and the library has the best semantics: async/await's semantics came essentially from the co module from npm, and almost nothing else came close to it in popularity. Its semantics were chosen as it was the simplest and soundest, though other mechanisms were considered. (The syntax choice was taken from C# due to similarity.) But this is the exception.

Sometimes, it's a few libraries dueling it out, like Moment and date-fns. The very heavy (stage 3) temporal proposal was created to ultimately subsume those with a much less error-prone framework for dates and times that's clearly somewhat influenced by the Intl APIs. This is still not the most common case, though.

Sometimes, it's numerous libraries offering the same exact utility, like Object.entries and Object.fromEntries both being previously implemented in Lodash, Underscore, jQuery, Ramda, among so many others, I gave up years ago even trying to keep track of the "popular" ones with such helpers. In fact, both ES5's Object.keys and all the Array prototype methods added from ES5 to today were added while citing this same kind of extremely broad library and helper precedent. CoffeeScript of old even gave syntax for part of that - here's each of the main object methods (roughly) implemented in it:

Object.keys = (o) ->
    (k for own k, v in o)

Object.values = (o) ->
    (v for own k, v in o)

Object.entries = (o) ->
    ([k, v] for own k, v in o)

Object.fromEntries = (entries) ->
    o = {}
    for [k, v] in entries
        o[k] = v
    o

Speaking of CoffeeScript, that's even inspired additions of its own. And there's been many cases of that and/or other JS dialects inspiring additions.

There's also other cases (not just temporal) where existing precedent was more or less thrown away for a clean sheet re-design. For one glaring example, iterables tossed most existing precedent. Anything resembling symbol-named methods are used by nobody else. Nobody had .throw() or .resume() iterator methods. .next() returns a special data structure like nobody else. (Most use "has next" and "get next and advance", Python uses a special exception to stop, and many others stop on null.) Library precedent centered around .forEach and lazy sequences, which was initially postponed with some members at the time rejecting it (this has obviously since changed). JS generators are full stackless coroutines able to have both yield and return values, but do did Python about a decade prior, so that doesn't explain away the modeling difference.

mlanza commented 3 months ago

It's not that I don't think signals are a staple of development. They most certainly are, at least for me. I just think you might get some pushback on what makes sense for the primitives.

JavaScript pipelines are also a necessity and what appeared as straightforward, because Babel had implemented F# pipelines long ago, ended up becoming so extremely fragmented in disagreement as to hit a standstill. And I see signals as far more sophisticated and varied than pipelines.

For example, take this statement from the proposal:

Computed Signals work by automatically tracking which other Signals are read during their evaluation.

This is one variety of how signals can be implemented. I don't like magic. When I was heavily into Ruby I grew to have a strong distaste for the amount of things which had to be understood as happening behind the curtain (e.g. magically). I came to prefer explicit.

So there are a lot of ways signals might be implemented. And if the way the primitives are implemented is distasteful to a group of devs who have other priorities/philosophies for how it should be done, they'll end up using their preferred third-party library. And native signals library will become an extraneous cost. It gets loaded, like it or not, because it's part of the runtime.

dead-claudia commented 3 months ago

It's not that I don't think signals are a staple of development. They most certainly are, at least for me. I just think you might get some pushback on what makes sense for the primitives.

For context, I myself coming in was hesitant to even support the idea of signals until I saw this repo and dug deeper into the model to understand what was truly going on.

And yes, there's been some pushback. In fact, I myself have been pushing back on two major components of the current design:

I also pushed back against watched/unwatched hooks on computeds for a bit, but since backed off from that

I've also been pushing hard for the addition a secondary tracked (and writable) "is pending" state to make async function-based signals definable in userland.

mlanza commented 3 months ago

I don't know what the community sentiment is, but is there a library in JS land which is already considered the de facto standard? Would it be RxJS?

If yes, wouldn't that library form the basis of this proposal? If no, doesn't the lack of a de facto standard seem to suggest a lack of consensus about what signals should be? I mean, why would this newish proposal/api more likely get it right over, say, a mature, well-known library?

I am not arguing against the proposal. I appreciate signals and use them in most UIs I write. I just struggle to understand how a new proposal is is likely to get right what one of the existing libraries hasn't already gotten right.

I recall there is a proposal for adding Observables to the runtime, which is directly related to signals. It's just that particular proposal offers a basic enough primitive/api to reach a consensus. If this one can likewise come up with the right primitives/api, great! I appreciate the effort.

The one thing the Clojure team had going for it when it authored core/async, which is CSP rather than FRP, but close enough, is it appeared to model what Golang had done. In the end, it wasn't design by committee. It was feature complete and so good enough. But then came spec and I guess it made mistakes, because was there spec v2? The point I'm trying to make is that you don't always know how solid a design is until you've had a chance to evaluate it in practice.

And since JS is baked into runtimes, whatever library is adopted becomes an irreversible choice. So this new library should be held as best of breed (by a community majority) after it's designed and used by those having tried the alternatives (RxJS, Bacon, xstream). If that happens, I guess it'd be ready for adopting into the core.

I don't think it need be robust (e.g. all the signal varieties offered by RxJS). It just needs to define/showcase the right primitives/api. The robustness can be tacked on with optional libraries. The aim of getting native primitives is in the optimization, not the robustness.

dead-claudia commented 3 months ago

@mlanza Welcome to the world of the average new stage 1 proposal, where everything is wildly underspecified, somehow both hand-wavy and not, and extremely under flux. 🙃

https://tc39.es/process-document/ should give an idea what to expect at this stage. Stage "0" is the wildest dreams, and stage 1 is just the first attempt to bring a dose of reality into it.

Stage 2 is where the rubber actually meets the road with most proposals. It's where the committee has solidified on a particular solution.

Note that I'm not a TC39 member. I happen to be a former Mithril.js maintainer who's still somewhat active behind the scenes in that project. I have some specific interest in this as I've been investigating the model for a possible future version of Mithril.js.

devmachiine commented 3 months ago

A single shared library isn't on its own a reason to do that. And sometimes, that library idiom isn't even the right way to go.

Good point, I agree. I can see the similarity between game designers and gamers who propose balances/changes which wouldn't benefit the game(rs) as a whole and/or have unintended consequences.

everywhere in control-oriented applications .. very common in hardware and embedded

Interesting. Especially the circuitry example! Because X exists in Y isn't enough justification on its own to include X in Z. I don't think javascript is geared towards those use cases, it's more in the domain of c/zig.

As for utility, it's not generally useful to most server developers. This is true. It's also of mixed utility to game developers. It is somewhat niche. But there's two points to consider: It would be far from the first niche proposal to make it in, and there's functionality even more niche than this. Atomics are very niche in the world of browsers.

Good point! I found the same to be true with the Symbol primitive. For years, I didn't really get it, but once I had a use case, I loved it.

I reconsidered some of the pro's of signals being baked into the language(or DOM api) which I stated Less code to ship Even if signals usage is ubiquitous, technically less bytes are sent over the wire, but practically not. Performance Technically yes, but practically? If a substantial portion of compute is taken up by signals processing, its probably a simulation or control-oriented application, and here I think it's out of domain scope for javascript again.

recall there is a proposal for adding Observables to the runtime, which is directly related to signals https://github.com/tc39/proposal-observable

There were similar concerns regarding the conclusion of not moving forward with the Observable proposal

I think there will be a lot of repeat discussion:

Why does this need to be in the standard library? No answer to that yet.
Where does this fit in? The DOM
Are there use cases in Node.js?
(examples of DOM apis that make sense in DOM and Node, but not in language)
Concerns about where it fits in host environments
Stronger concerns: will this be the thing actually _used_ in hosts?

I can appreciate the standardization of signals, but I'm not convinced that tc39 is the appropriate home for signals. The functionality can be provided via a library, which is much easier to extended and improve across contexts.

dead-claudia commented 3 months ago

Technically yes, but practically? If a substantial portion of compute is taken up by signals processing, its probably a simulation or control-oriented application, and here I think it's out of domain scope for javascript again.

@devmachiine You won't likely see .set show up on performance profiles, but very large DOM trees (I've heard of trees in the wild as large as 50k elements, and had to direct someone with 100k+ SVG nodes to switch to canvas once) using signals and components heavily could see .get() showing up noticeably.

But just as importantly, memory usage is a concern. If you have 50k signals in a complex monitoring app (say, 500 items, with 15 discrete visible text fields, 50 fields across 4 dropdowns, 20 error indicators, and 5 inputs), and you can shave off an average of about 20 bytes of each of those signals by simply removing a layer of indirection (2x4=8 bytes) and not allocating arrays for single-reference sets (2x32=64 bytes per impacted object, conservatively assumes about 20% are single-listener + single-parent), you could've shaved off around entire entire megabyte of memory usage. And that could be noticeable.

justinfagnani commented 2 months ago

To me, the biggest advantage of this being part of the language is interoperability.

If you want multiple libraries and UI components to interoperate by being able to watch signals in a single way, a standard feature is the only way to accomplish that. It's infeasible to have every library use the same core signals library, and eliminate all sources of duplication (from package managers, CDNs, etc) which would bifurcate the signal graph.

mlanza commented 1 month ago

To me, the biggest advantage of this being part of the language is interoperability.

If you want multiple libraries and UI components to interoperate by being able to watch signals in a single way, a standard feature is the only way to accomplish that. It's infeasible to have every library use the same core signals library, and eliminate all sources of duplication (from package managers, CDNs, etc) which would bifurcate the signal graph.

I agree with this sentiment. So to restate, perhaps a little differently, just give us the optimized primitives, not a replete, one-size-fits-all signal library. Built the primitives in such a way that every well-known library (RxJS, BaconJS, etc.) can be built from them.

In this way, even the current library on offer could be built from these primitives. This preserves a good deal of choice about how do things. Even Observable appears to have gone this route.

justinfagnani commented 1 month ago

So to restate, perhaps a little differently, just give us the optimized primitives, not a replete, one-size-fits-all signal library.

Well, I also think that the built-in APIs should be as ergonomic as reasonably possible. We shouldn't require that a library be used to get decent DX.

I personally think the current API is near a sweet spot because it makes the common things easy (state and computed signals), and the complex things possible (watchers). Needing utility code for watchers makes sense, but IMO basic signal creation and dependency tracking should be usable with the raw APIs.

In this way, even the current library on offer could be built from these primitives

I struggle to think of what lower-level primitives could be useful. You need centralized state for dependency tracking. Maybe you could separate a signals local state from its tracked state - say, State doesn't contain it's own value - but you still need objects of some sort to store dependency and dirtiness data. I don't know what a lower-API would even get you over the current State and Computed.

mlanza commented 1 month ago

Well, as an example, I implemented my own signal, called an atom, after Clojure's atom. It's pretty close in spec compared to Clojure's. And it's api is trivially small. It's the basis of everything I do in my library.

I've used it for a decade and it does everything that's needed to build reactive apps. I've built every conceivable kind of app from games to enterprise UIs from this one, tiny primitive. My point is not much is actually needed. Robustness is tackling every use case. Minimalism is tackling the main use cases but easily affording as much robustness as is needed with just a little extra work.

justinfagnani commented 1 month ago

@mlanza To be concrete, what would your Atom implementation look like under the current API, vs an alternative that may be lower-level? Is State a hinderance?

mlanza commented 1 month ago

First, what do you mean by is state a hinderance?

My understanding of reactives is that you have source reactives (ones which house state) which are both read/write (e.g. an atom) and sink reactives which are read only and derive their state from some other reactive (of either variety). Thus, state, at the source, is a necessary starting point, as you can't have either variety downstream from no state.

justinfagnani commented 1 month ago

First, what do you mean by is state a hinderance?

I mean, is the class Signal.State more difficult to use to implement your Atom than whatever a lower-level primitive might be? Can we compare the current API against a hypothetical one?

mlanza commented 1 month ago

I guess I'm saying that this proposal is good enough if its primitives can be easily adapted for use in any existing popular signal library. I'm noting that the various signal libraries all have their own caveats and some variation in how they do things and I like that variation, how a problem is seen from different angles, as opposed to presenting a single, opinionated best practice solution for everyone who does signals.

Because, if this proposal gets the basics right, and it can be easily adapted for use within any popular library, then I trust I could easily adapted for use as the core of my own library. It may very well do that in its current form and, if it does, then I commend it.

I don't want to have take a deep dive to understand a library's primitives. If a deep dive is required to understand things like "automatic dependency tracking" then the primitives aren't primitive enough. Usually magic isn't free and I prefer transparent and explicit to things magically done on my behalf. I just want to preserve some option for going this way or that (as different libraries already do) as opposed to saying, this is the one true way. As I said, if your primitives are basic enough and complete enough, you get all that. So the authors can, if they want, assess whether this bar has been met.

justinfagnani commented 1 month ago

I'm just trying to get concrete here. Are these primitives in the current proposal minimal and complete? What would be lower level?

You seemed to be saying that the current API could be built from lower-level primitives. So I'm asking, what are those lower-level primitives? And how does your library look like implemented with the current proposed API vs those lower-level primitives?

mlanza commented 1 month ago

I'm saying that you'll know the primitives are low-level enough if they are suitable as primitives from which any of the popular signal libraries could be derived. I am not going to spend the many, many hours which'd be necessary to learn and evaluate the proposal as it exists in order to, politely, answer your question to your satisfaction. I know that anyone who puts up a proposal for t39 easily spends hundreds of hours, to reach a mature, well-ratified consensus.

I am offering a single requirement which the authors can, if they wish, and if they feel it worthy, consider. That's all. By putting up the proposal, they've already enlisted to consider/evaluate and do the heavy lifting.