Observable object

[luwes]

Some research on whether an observable object should be added which is more in line with how Vue, Mobx and Solid use a proxy to the accessor of the observable.

This provides some advantages:

• Simpler to get and set, state.color instead of color().
• Easy JSON stringify

Disadvantages:

• Always need to be called from the state object, users might find it strange the value can't be cached without losing the reactivity.

[brucou]

I think you have summarized well the main tradeoff here. Leaning in on the subject, I believe writing correct code or facilitating writing correct code is the first order of priority, and I would go for color(). More than facilitating writing code (optimize for code writing and reading), I would facilitate writing correct code (optimize for debugging). For a new library, I always spend most of my time debugging my mistakes, so if I can avoid making those mistakes in the first place that is cool. I have started using Svelte a few months ago, and there is the same gotcha with its reactivity system, and I fell into all the holes I could find, because the syntax leads you to believe that you can some stuff that you cannot.

Now from an adoption point of view, to make it easy for developer to get up on the wagon, it is true that proxy magic can help and there is a growing list of libraries resorting to it (Immer is a good example of that). Not easy to decide which is best.

[luwes]

thanks for chiming in @brucou, I appreciate it! those are valid points, I'm leaning towards keeping the current behavior without the proxy magic for v1. it's easier to add later, than remove or deprecate.

[ryansolid]

There are 2 problems with transparent POJO reactivity as far as I see it:

1. Reactivity is based on property access state.color. People have to be conscious of de-structuring and variable assignment. Now it isn't like using observables/signals you don't need to be aware of where you call your accessor color() but there is a defined syntax with functions. So people still need to understand reactive context and access, so proxies do not require less knowledge. The biggest issue here is people's instinct is to do the exact opposite they should be in these cases. If they want to preserve reactivity they should just assign access to a thunk:

   const color = () => state.color

   Now you could have just started with an observable but a proxy is convertable to essentially observable at any time by wrapping in a function. This is what differs a fine grained library from something like Svelte that relies on the compiler as you have more control because your syntax isn't limited.

2. Loss of context in transfer in state partials. Let's say we start state.user and we pass it to child who now has user or perhaps props.user. Do we know if it is reactive if someone access user.firstName somewhere. I'd argue that on get access where generally you want updates to occur this is acceptable. However, what if someone writes user.firstName = "Jake"? Do we have any idea of what we are doing? Does anyone know where this change is happening? Maybe it was a typo in a comparison. Now passing nested observables around can have this problem, but setting is very explicit.

There is two key benefits:

1. At some point you will find yourself mapping observables with objects/arrays with observables in it, and observables in those and so on. This is a constant overhead when pulling models in from the server etc. Whereas a Virtual DOM library can sort of map as it goes down and diff each time, these systems are built on mutation so you need to keep your references. So you tend to reactive wrap, and then map that to view. You end up mapping twice and it's eager. With proxies you can lazily handle mapping and remove all that overhead. In practice it means that certain domains of problems like data snapshots/time travel annoying data blasting benchmarks, etc can be handled fairly trivially.
2. Potential for explicit tree mutation control. It is not obvious but if the proxy is self nesting and you don't need to map etc the potential to pull the setter out of the whole tree is incredibly powerful trick to ensure uni-directional control. With these libraries synchronous batching capabilities you get the same control benefits of say Redux without resorting to reducers. This means improved traceability. Something that becomes more valuable when dealing with large portions of shared state. Solid uses this approach and I think MobX State Tree does something similar.

So in summary, I'd say it isn't a replacement necessarily just different. Admittedly I was pretty skeptical of using Objects like this 4 years back, and that was after using these fine grained libraries in production already for 6 years. So I can respect different opinions. But I also think they open up a few (optional) doors that can be pretty attractive depending. I feel like this is a whole subject on its own and I probably wouldn't touch this one until you are ready to work through all the tradeoffs and decide how your library wants to treat these. These aren't as clearly defined. I played with several variations before I landed on what I use now. And if the discussions around Vue RFC are indicator, this is definitely an area where opinions come out in force.

[kethan]

Any simple idea to do this now? My team is getting confused to call a function to update anything similar to update via proxy or using the variable for changes?

[luwes]

@kethan for the moment this won't be added to the Sinuous core as the current approach with function calls is more low-level. Proxies are also not IE11 compatible, I think it should work with Object.defineProperty, not 100% sure. I believe this behavior can still be build on top of Sinuous observable or it's even possible to use a fully different reactive library by using Sinuous's internal api. cheers

[kethan]

@kethan for the moment this won't be added to the Sinuous core as the current approach with function calls is more low-level. Proxies are also not IE11 compatible, I think it should work with Object.defineProperty, not 100% sure. I believe this behavior can still be build on top of Sinuous observable or it's even possible to use a fully different reactive library by using Sinuous's internal api. cheers

Is it possible to throw some light or a simple snippet to do so.

[kethan]

@kethan for the moment this won't be added to the Sinuous core as the current approach with function calls is more low-level. Proxies are also not IE11 compatible, I think it should work with Object.defineProperty, not 100% sure. I believe this behavior can still be build on top of Sinuous observable or it's even possible to use a fully different reactive library by using Sinuous's internal api. cheers

import { o, html } from "https://unpkg.com/sinuous?module";

function cloneObject(obj) {
    var clone = {};
    for (let prop in obj) {
        if (obj[prop] != null && typeof obj[prop] === "object")
            clone[prop] = cloneObject(obj[prop]);
        else {
            const ob = obj[prop];
            if (ob.$o) clone[prop] = o(ob());
            else clone[prop] = ob;
        }
    }
    console.log(clone);
    return clone;
}

function proxy(state) {
    let clone = cloneObject(state);
    for (let prop in clone) {
        if (true) {
            Object.defineProperty(state, prop, {
                get() {
                    return clone[prop].$o ? clone[prop]() : clone[prop];
                },
                set(value) {
                    clone[prop].$o ? clone[prop](value) : (clone[prop] = value);
                }
            });
        }
    }
}

let counter = {
    count: o(0)
}

proxy(counter);

const view = () => html` <div>Counter ${() => counter.count}</div> `;

document.body.append(view());
setInterval(() => {
    counter.count = counter.count + 1;
}, 1000);

Thank you @luwes I have implemented with your idea given. But how can I use this

without ${() => counter.count } like ${counter.count}

[ryansolid]

You don't. The problem is using a proxy doesn't save you from having to defer evaluation until it is in the proper reactive context. So you still need to wrap it in a function. While it is perhaps convenient for simple assignment or read syntax you still need to wrap your computeds. See problem 1 in my response above.

[luwes]

@ryansolid is right, it's impossible without a compiler.

this is a related issue where I was thinking of adding this as an option to compiled only templates. https://github.com/luwes/sinuous/issues/78

Surplus and Solid also achieve this with their compiler.

[mindplay-dk]

I came across this issue for a different reason than laid out in the original post, I'd like to share.

I've been thinking about dependent state, and don't currently see a good way to deal with this using observables.

Consider the following use-case: a simple numeric input (or slider) with a min and max value, which can change. Since the input value is not allowed to be outside the min and max, updating the individual states can create problems.

For example, if the current state is min=0, max=10, value=5, and we want to change the state to min=0, max=20, value=15, we can't just make these changes individually in any order - since, if we were to update the value first, it would momentarily be greater than the max, meaning it's out of range; meaning, our overall model is temporarily in an invalid state.

If that doesn't sound like a big deal, consider the implications of, for example, drawing a custom slider UI, where the value is outside the permitted range - you can clip the value, of course, but this is accidental complexity: we shouldn't have to deal with an overall state that isn't valid in the first place. I'm using a trivial example here, but hopefully this illustrates that there's a whole class of similar problems to consider for more complex cases, which would require more complex workarounds.

In React, you don't have this class of problems, since state updates are applied as a set - so in the case above, the min, max and value states are updated in a single transaction, e.g. setState({ min=0, max=20, value=15 }), which means there's no momentary overall state where the value might not be valid; you don't have to consider, say, where to draw the handle on a custom slider.

In Sinuous, I could of course get around this by creating a single observable containing an object - so the dependent values can only be updated as a whole. The problem with that approach, is the granularity of the UI updates: if you use a single observable, the min, max and value inputs will all update anytime one of them changes. Again, this may not sound like a very serious example, but I'm just trying to demonstrate that there's a class of problems that exists here - to use an extreme example, consider the consequences of putting your entire app state in a single observable object.

While very intuitive and convenient on the surface, the API which uses getters and setters doesn't really work for cases with dependent state, I think - since the setters are just updating individual states.

Something like this might work better:

const state = observable({ min: 0, max: 10, value: 5 });

// transactional update:

state({ min: 0, max: 20, value: 15 });

// partial update:

state({ value: 2 });

// transactional (destructured) read:

const { min, max, value } = state();

// individual read:

const value = state.value();

It's not quite as intuitive, but should feel familiar to someone who is comfortable with observables - the only subtle deviation here is partial updates, but those should be familiar to anyone who's worked with React state, for example.

Any thoughts? 🙂

[mindplay-dk]

(note that this would probably need a different constructor than observable() - since this would be a breaking change.)

[leeoniya]

which means there's no momentary overall state where the value might not be valid

this is also a frequent criticism of web components, in that custom elements provide no mechanism to batch together updates of individual attrs.

[ryansolid]

S.js has freeze, MobX has actions. Vue has a similar mechanism under the hood. This batching is pretty common. In this sort of library. In my state proxies in Solid I do batch this way automatically in the setState method. But you don't need objects to do this although it does make it more succinct.

Sinuous has actions or transactions if I remember correctly.

[luwes]

yup Sinuous has transactions, https://github.com/luwes/sinuous/tree/master/packages/sinuous/observable#transaction

related issue https://github.com/luwes/sinuous/issues/21

[luwes]

@leeoniya I've encountered this issue as well with custom elements.

I put in the await promise technique for the Swiss library to debounce rendering, not sure if that's what you mean? https://github.com/luwes/swiss/blob/master/packages/swiss/src/updating-element.js#L23

I guess it's used in most libs that render something.

I was kinda surprised lit-html reflects attributes not synchronously, the reflection is also debounced if a property is set if I remember correctly.

[mindplay-dk]

Do transactions work across multiple observables?

The inline documentation doesn't seem to indicate this:

Creates a transaction in which an observable can be set multiple times but only trigger a computation once.

I also don't see a test covering transactions across multiple observables?

If it does work, we should probably update the docs and add a test, maybe an example too.

I'm assuming it does work, since you mentioned it in this context - if so, that's a solved problem for me, something I've been wondering about for a while actually.

But then, I wonder, what's the requirement? What is it you want from this feature? If it's just for convenience, in my minimalistic mindset, I'd say "no thanks" and suggest we opt for better documentation instead. 😉

[luwes]

good point @mindplay-dk, it doesn't work across multiple observables I believe.

sorry I've been working on a different project for a while, nothing comes up right away but I feel this should be possible to accomplish but maybe not in a very convenient way.

[luwes]

if you use a single observable, the min, max and value inputs will all update anytime one of them changes.

this is true but I think in most situations negligible.

also was just thinking the diffing that React does must be in some step synchronous where one attribute will be updated and then the next. it's not possible to update attributes all at once.

[mindplay-dk]

this is true but I think in most situations negligible.

also was just thinking the diffing that React does must be in some step synchronous where one attribute will be updated and then the next. it's not possible to update attributes all at once.

Well, Object.assign 😉

But yes, that's all I meant by "transactional": in a single, blocking operation - meaning, no display updates get triggered while doing individual updates, so your view-logic never needs to consider temporary, invalid states.

It is negligible in many situations, but it's not a safe assumption, and the sort of thing that leads to subtle bugs (garbled UI) or error (division by zero, etc.) in cases where you didn't consider all the possible unexpected/invalid temporary states.

As @leeoniya pointed out, it's also a common complaint about web-components. I personally found it leading me into accidental complexity, besides being very distracting to think about while you're trying to focus on the real requirements for a component. It's the main reason I don't want web-components in my own projects and still prefer something like Sinuous or Preact.

Anyhow, I understand you have other priorities at the moment, so I might attempt a PR. 😊

[mindplay-dk]

good point @mindplay-dk, it doesn't work across multiple observables I believe.

Turns out, it does! 😀

So I think we're actually good on that front - being able to batch updates in a blocking manner was really the only reservation I've had about Sinuous thus far. Sorted! 😏

Another thing that wasn't covered by tests is nested transactions, and here I found the results a bit surprising, as it appears all changes are held until the outermost transaction completes.

I'm not actually sure "transaction" is the right way to describe what this function does? Yes, it batches things, similar to transactions in databases and file-systems, etc. - but these "transactions" don't have a scope... pending changes aren't visible from anywhere.

This is probably fine for batching changes at a single level - but I'm not sure nested "transactions" are really meaningful?

Since a function can't know if it's already in a transaction, it might expect to batch some changes, and then, say, calls another observable for some derived state that involves one or more of the same observables it just updated. Let's say it uses this result to load an image or whatever. The result will be unpredictable - if you're not already in a transaction, it'll work, but if you're already in a transaction, your updates simply didn't happen yet, and you end up loading the same image.

So again, I'm not sure this feature makes sense? I wonder if it should be disallowed? I'm not sure what's worse - getting an exception or just getting a different result than what you intended. 🤔

[ryansolid]

Nested batching like this typically desired because people generally set them up as guards. If you expect your expression to be batched and children batch you don't expect inconsistency in your batch. Although not an example here, I batch my proxy update setter calls. If someone chooses to batch multiple of those calls the whole thing should be part of the outer batch. As long as the implementation reads the pending value when reading an observable during the pending batch execution you should be good. Maybe I'm misunderstanding the case or the Sinuous implementation isn't accounting for reads, but in general there shouldn't be issues unless you are trying to read from the result of a computed inline outside of your inner batch but inside the outer one.

[mindplay-dk]

I guess, in my own example, you could fix this by making sure updates to that image are also reactive - rather than triggering the image load explicitly, make it an effect, so that the image loading happens whenever the outermost transaction finishes. In that case, there's no problem.

I think that's the only real issue I have with observables: at a glance, it looks like "just javascript" - but there is some learning curve here, as I'm sure some users will intuitively try to do things like what I described in my previous comment, because they can, and it works superficially... but reactivity really works best, and in some cases only works correctly, if you fully embrace the idea and make every aspect of the UI fully reactive.

(This might be an important point to touch on in a tutorial.)

[luwes]

❤️