Question: Subscribing to nested changes at a parent level

[jmchuster]

Hi avkonst,

Amazing library you've built here. It looks exactly like the type of thing I was imagining. I'm going through evaluating libraries to try to get to a less hacky way of implementing state.

I had a question on a specific use case that's not obvious to me if hookstate would handle well (or could have a plugin written to handle).

I have a large recursive nested structure, basically a workflow of questions and answers which lead to more questions then more answers and so on. I've built a hacked-together solution with regular react components (well, preact), where the top-level component has the entire tree as a state variable, passing down sections of the tree to its children, and then each nested component has an oninput that mutates its section of the tree directly and then calls forceUpdate(). This works because the tree of Components mirror the tree of the state, so forceUpdate() correctly cascades for the corresponding subtree. I'm sure many would cry at this implementation, but it works well enough.

In hookstate, it looks like nested combined with the usage of scoped states would work well enough as a replacement here. The subtree would get passed in to each component, and it would be able to set on the scoped state oninput, which should then update the state and trigger a render.

Then, I support undo/redo functionality, by changing the top-level state tree into a proxy, so when anything is set or delete, I store a snapshot of the whole tree, and then undo/redo buttons just setState the whole tree with the snapshot, so everything renders as normal.

This is then where I'm not sure how I would implement this snapshot/apply functionality in hookstate. I think what I would want is some way to know at the top-level (or maybe even any nested level) that there is a change down the line, so that I can know to store a snapshot. There is a Touched plugin demo that shows the parent knows when it was touched, but do those just store a one-time flag for the first change? Or is there some way to subscribe to any changes? This also makes me think about things like time travel, though I think maybe your library specifically chose to eschew this functionality.

Maybe I'm thinking about this the wrong way, and it would just be most straightforward to add a transform for saveSnapshot and have the Component oninput function just call that after state set.

Any thoughts/advice?

[avkonst]

Your requirements are easy to handle with Hookstate.

1. Here is the example of recursive tree structure with scoped states for each field (it rerenders only the updated field and JSON dump component for debugging purposes) https://hookstate.js.org/local-complex-tree-structure

2. To achieve historical state tracing you need to watch for a state change on root level. There are 2 ways. a) you can replace JsonDump from the example with your listener of changes and store snapshots in another state variable (Hookstate or not). This is not recommended approach because you will set the other state on rerender, which is not advised. b) you can implement Hookstate plugin, which can be generic by the way. I needed complete version tracking at one stage but moved on without it. The plugin would subscribe to changes and store the state in its own structure and would expose the API to retrieve versions. Regarding the plugin implementation read below.

You can implement the plugin like the following (writing without compiler check):

import { Plugin, StateLink, StateValueAtPath } from '@hookstate/core'

interface HistoryExtensions {
    getSnapshots(): any[] // I do not bother with types here, although it is possible to make it fully typesafe with generics
    // you can also make the interface friendlier,  with fetch by version ,etc.
}

const HistoryPluginID = Symbol('HistoryPlugin')
function History(self: StateLink<StateValueAtPath>): HistoryExtensions;
function History(): Plugin;
function History(self?: StateLink<StateValueAtPath>): Plugin | HistoryExtensions {
    if (self) {
        return self.with(HistoryPluginID)[1] as HistoryExtensions;
    }

    const snapshots: any[] = [];
    return ({
         id: HistoryPluginID,
         create: (l) => {
             snapshots.push(l.value) // capture the initial state
             return ({
             onSet(arg) {
                 console.log('Changed field at path', arg.path)
                 console.log('Changed from', arg.previous)
                 console.log('Changed to', arg.value)
                 snapshots.push(JSON.parse(JSON.stringify(arg.state))) // deep object clone (you can use lodash deepclone instead) 
                // deep clone is not efficient for large objects,
                // you can think of other ways of tracing capturing the snapshots. 
                // for example, recording changesets using arg.path and arg.value
                // and capture snapshots only 1 out of 100 state updates (this is what I do in one of my persistence plugins - works very efficiently)
             },
             getSnapshots() {
                 return snapshots;
             }
             } as Plugin & HistoryExtensions)
         }
    })
}

Now you can use it like this:

// create the plugin
const myState = createStateLink(...).with(History);

// use as usually
myState.set(...) 

// get snapshots
History(myState).getSnapshots()

Hope it helps. Let me know how it goes.

[jmchuster]

ah, that's really powerful, that the onSet has access to the path as well. Is there documentation available on the plugin api?

To then render the list of available snapshots, is use stateLink required? as in

const SnapshotsComponent = () => {
    const stateLink = useStateLink(myState);
    return History(stateLink).getSnapshots().map { it => html`<div>Snapshot: </div>` };
}

vs

const SnapshotsComponent = () => {
    return History(myState).getSnapshots().map { it => html`<div>Snapshot: </div>` };
}

Namely, what difference does it make calling a plugin with StateInf as the argument as opposed to StateLink as the argument?

[avkonst]

Best docs for plugin API is the typescript declarations at this stage. Do you use typescript? there are 2 callbacks at this stage - onSet and onDestroy. Just inject your custom plugin and see what object you receive as the arg on various state changes.

Regarding StateInf vs StateLink. In the recent version StateInf is deprecated and it is all StateLinks, where it can be a pointer to a global state or local state or scoped state. There is newer documentation under construction, which I have not published yet, because it is really long to write docs. But the question you asked is already covered by the newer docs. You can read it now, just self-serve it by going to:

git clone https://github.com/avkonst/hookstate
cd hookstate
cd hookstate.js.org/docs2
yarn install
yarn start

and open the browser at the address suggested. I would like to receive your feedback about the newer documentation. I believe it will cover the question you are asking.

[avkonst]

BTW, this History(stateLink or myState).getSnapshots().map { it => html<div>Snapshot: </div> } would not work as you expect, because the returned variable is an array variable and not a state being watched.

You have got 2 ways to trigger rerender when the snapshots variable is changed.

The first is

import { Downgraded } from '@hookstate/core'

const SnapshotsComponent = () => {
    const stateLink = useStateLink(myState).with(Downgraded);
    stateLink.get() // mark it as used, so a change on any nested state triggers reerender of this component
    return History(stateLink).getSnapshots().map { it => html`<div>Snapshot: </div>` };
}

or the second is to make the snapshots variable as the state itself (see how I replaced any[] by StateLink<any[]>):

import { Plugin, StateLink, StateValueAtPath } from '@hookstate/core'

interface HistoryExtensions {
// !!! change here:
    getSnapshots(): StateLink<any[]> // I do not bother with types here, although it is possible to make it fully typesafe with generics
    // you can also make the interface friendlier,  with fetch by version ,etc.
}

const HistoryPluginID = Symbol('HistoryPlugin')
function History(self: StateLink<StateValueAtPath>): HistoryExtensions;
function History(): Plugin;
function History(self?: StateLink<StateValueAtPath>): Plugin | HistoryExtensions {
    if (self) {
        return self.with(HistoryPluginID)[1] as HistoryExtensions;
    }

// !!! change here:
    const snapshots: StateLink<any[]> = createStateLink<any[]>([]);
    return ({
         id: HistoryPluginID,
         create: (l) => {
             snapshots.push(l.value) // capture the initial state
             return ({
             onSet(arg) {
                 console.log('Changed field at path', arg.path)
                 console.log('Changed from', arg.previous)
                 console.log('Changed to', arg.value)
 // !!! change here:
                 snapshots.merge([JSON.parse(JSON.stringify(arg.state))]) // deep object clone (you can use lodash deepclone instead) 
                // deep clone is not efficient for large objects,
                // you can think of other ways of tracing capturing the snapshots. 
                // for example, recording changesets using arg.path and arg.value
                // and capture snapshots only 1 out of 100 state updates (this is what I do in one of my persistence plugins - works very efficiently)
             },
// !!! change here:
             onDestroy() {
                  // it is optional for the plain variable like the array without native resource bound, but add it for the completeness of the picture 
                  // the idea is simple - when the main state is destroyed, we want to destroy the state created by the plugin
                  snapshots.destroy()
             }
             getSnapshots() {
                 return snapshots;
             }
             } as Plugin & HistoryExtensions)
         }
    })
}

and use it like the following:

const SnapshotsComponent = () => {
    const stateLink = useStateLink(myState);
    const snapshotsLink = useStateLink(History(stateLink).getSnapshots()) // subscribes to changes in the snapshots state variable
    return snapshotsLink.value.map { it => html`<div>Snapshot: </div>` };
}

[jmchuster]

I've been working at this all week. Lots of good learnings and insights that I'm hoping to create a series of blog articles about once I'm all done.

I'm now trying to dig into optimizing the performance and renders. Could you help clarify the scoped state for me some?

For example, I have a calendar represented by a state object that has a couple of properties. I'd imagine that a component that is only concerned with one property shouldn't care about the other.

Could you help explain why this doesn't prevent re-rendering when other properties change:

const showWeekends = useStateLink(globalStateLink.nested.showWeekends).value;

but this does:

const showWeekends = useStateLink(globalStateLink, StateMemo(it => it.nested.showWeekends.value));

Also noticed that it seems to "stabilize" after a while, so I see it stopped re-rendering on unrelated changes after 3 touches.

I thought that the idea of scoped state was to prevent re-renders when dealing with nested state, so changing other properties like globalState.events shouldn't cause a component that only relies on globalState.showWeekends to re-render.

Is it that I'm using globalStateLink? Do i need to be passing it down from the parent? In that case, why wouldn't that just end up triggering a re-render on the parent, which then triggers re-renders on all the children anyways? Anyways, I think I'm missing something here about what useStateLink is really doing when it wraps other state links.

Also note that my example above is a bit simplified. I'm actually trying to do some wrapping around the state, so maybe that is what is causing the issue like this, but I still think it would work since it should still just be like useStateLink(useStateLink(createStateLink({}))):

const createModelState = {
    const transform = (state) => {
        getGlobalStateLink() { return state; }, // exposed for the testing above
        isShowWeekends() { return state.nested.showWeekends.get(); },
        setShowWeekends(show) { state.nested.showWeekends.set(show); }
        ....
    });
    return createStateLink({}).wrap(transform);
}
const ModelContext = createContext(createModelState());
const useModelState = () => {
    return useStateLink(useContext(ModelContext));
};

function Days() {
    const state = useModelState();
    const showWeekends = state.isShowWeekends();
    return html`<div class=${showWeekends ? 'show-weekends' : ''} />`;
}

[avkonst]

Here is the complete matrix of use cases and re-render results.

Let's say there are the following components:

const globalState = createStateLink({ A: 0, B: 0 }) // global state with 2 nested states

// Example 1 every child component consumes the state, but parent does not
function Example1_Parent() {
    return <><Example1_ChildA /><Example1_ChildB /></>
}
function Example1_ChildA() {
    const state = useStateLink(globalState)
    return <p>{state.value.A}</p>
}
function Example1_ChildB() {
    const state = useStateLink(globalState)
    return <p>{state.value.B}</p>
}

// Example 2 parent component consumes the state and passes nested to children, children do NOT use scoped state
function Example2_Parent() {
    const state = useStateLink(globalState)
    return <><Example2_ChildA state={state.nested.A}/><Example2_ChildB state={state.nested.B}/></>
}
function Example2_ChildA(props: { state: StateLink<number> }) {
    const state = props.state
    return <p>state.value.A</p>
}
function Example2_ChildB(props: { state: StateLink<number> }) {
    const state = props.state
    return <p>state.value.B</p>
}

// Example 3 parent component consumes the state and passes nested to children, children USE scoped state
function Example3_Parent() {
    const state = useStateLink(globalState)
    return <><Example3_ChildA state={state.nested.A}/><Example3_ChildB state={state.nested.B}/></>
}
function Example3_ChildA(props: { state: StateLink<number> }) {
    const state = useStateLink(props.state)
    return <p>state.value.A</p>
}
function Example3_ChildB(props: { state: StateLink<number> }) {
    const state = useStateLink(props.state)
    return <p>state.value.B</p>
}

// Example 4 to explain StateMemo
function Example4_SumWithoutStateMemo() {
    const sumState = useStateLink(globalState.wrap(l => l.value.A + l.value.B))
    return <>{sumState}</>
}
function Example4_SumWithStateMemo() {
    const sumState = useStateLink(globalState.wrap(StateMemo(l => l.value.A + l.value.B)))
    return <>{sumState}</>
}

const incrementA = () => globalState.nested.A.set(p => p+1)
const derementB = () => globalState.nested.B.set(p => p-1)
const incrementA_derementB = () => globalState.merge(p => ({ A: p.A + 1, B: p.B - 1 }))

Y - component rerenders on state change X - component does not rerender on state change P - component rerenders because the parent is rerendered

rerender on actions:	incrementA	decrementB	incrementA_decrementB
Example1_Parent	X	X	X
Example1_ChildA	Y	N	Y
Example1_ChildB	N	Y	Y
Example2_Parent	Y	Y	Y
Example2_ChildA	P	P	P
Example2_ChildB	P	P	P
Example3_Parent	N	N	N
Example3_ChildA	Y	N	Y
Example3_ChildB	N	Y	Y
Example4_SumWithoutStateMemo	Y	Y	Y
Example3_SumWithStateMemo	Y	Y	N

Now answering your question ("Do i need to be passing a state down from the parent? In that case, why wouldn't that just end up triggering a re-render on the parent, which then triggers re-renders on all the children anyways?"):

• Consume the state in a component which is best suited for this to deliver the simplest and cleanest code. For example, if you render a table from a single state with rows as substates, consume the state in the parent component and pass row states to children components (Example 2 above). This will make your code simpler (and faster) than if you would go with approach consuming the state by each row (Example 1 above)
• Use scoped state (move from Example 2 to Example 3) when you notice nested state changes cause expensive and unnecessary re-rendering of the parent component
• Use StateMemo for nothing more than an aggregated value, like a sum. Do not use StateMemo otherwise (I am thinking to get rid of StateMemo in the version 2 as it actually can be replaced by React.memo really, but I have not decided)

Hope it helps. Let me know.

PS: also, have you tried new documentation? I think it clearly explains the nested and scoped state differences. There is also online demo which shows the difference when the scoped state is active/inactive: https://hookstate.js.org/demo-todolist/

[avkonst]

Also notice that the following components are identical in performance and re-rendering behaviour:

function Example1_ChildA() {
    // uses the global state
    const state = useStateLink(globalState)
    // and dives to the nested value later
    return <p>{state.value.A}</p>
}
function Example1_ChildA() {
    // dives to the nested state first and after uses it
    const stateA = useStateLink(globalState.nested.A)
    return <p>{stateA.value}</p>
}

So, do not bother with particular choice, use the one which delivers the cleanest code in a given situation. It can be different in different cases. All is valid and same great performance.

[avkonst]

Also, you do not need to use Context to deliver states. Just do this instead:

const createModelState = {
    const transform = (state) => {
        getGlobalStateLink() { return state; }, // exposed for the testing above
        isShowWeekends() { return state.nested.showWeekends.get(); },
        setShowWeekends(show) { state.nested.showWeekends.set(show); }
        ....
    });
    return createStateLink({}).wrap(transform);
}
const ModelState = createModelState();
const useModelState = () => {
    return useStateLink(ModelState);
};

function Days() {
    const state = useModelState();
    const showWeekends = state.isShowWeekends();
    return html`<div class=${showWeekends ? 'show-weekends' : ''} />`;
}

[jmchuster]

Thanks so much for the detailed write up. I'll spend some time to go through it and try it all out.

I've been going back and forth between the new documentation and the old. The todo demo in the new documentation shows the effect I wanted, but I haven't yet fully internalized what's the difference between that and mine.

I used Context to pass down the state because I wanted to be able to wrap the components with alternative test double versions of the ModelState.

[jmchuster]

I think some of my confusion was that I was using something akin to Example1, but still seeing a couple renders. But it seems like it only happened a couple times initially after setting some other states (not consistently), and then stopped re-rendering once it "stabilized". I'll work on trying to get an example to reproduce. Have you seen such behavior before?

Question A:

Could you explain why Example2_Parent() and Example3_parent() have different rendering behaviors even though they're the same code? How does the child influence the rendering of the parent?

What would happen if the code was like

// Example 5 parent component consumes the state and passes nested to children, child A USE scoped state but child B DOES NOT
function Example5_Parent() {
    const state = useStateLink(globalState)
    return <><Example5_ChildA state={state.nested.A}/><Example5_ChildB state={state.nested.B}/></>
}
function Example5_ChildA(props: { state: StateLink<number> }) {
    const state = useStateLink(props.state)
    return <p>state.value.A</p>
}
function Example5_ChildB(props: { state: StateLink<number> }) {
    const state = props.state
    return <p>state.value.B</p>
}

Would you then get

Example5_Parent     N   Y   Y
Example5_ChildA     Y   P   P
Example5_ChildB     N   Y   Y

Question B:

Does it matter that you use the value or not? Does it only matter which items get referenced via nested? If I set A, which of these should render?

function A() {
    const state = useStateLink(globalState)
    return <p />
}
function B() {
    const state = useStateLink(globalState)
    const value = state.value
    return <p />
}
function C() {
    const state = useStateLink(globalState)
    const valueA = state.value.A
    return <p />
}
function D() {
    const state = useStateLink(globalState)
    const nested = state.nested
    return <p />
}
function E() {
    const state = useStateLink(globalState)
    const nestedA = state.nested.A
    return <p />
}
function F() {
    const state = useStateLink(globalState)
    const nestedAValue = state.nested.A.value
    return <p />
}

In my mind I'm imagining that components should only re-render if a) value is used and that value has changed b) nested is used and that ref has changed So, in the example above, I'd guess that only B, C, and F render when we set the value. Is that the right way to think about it, or should I be thinking about it another way?

[avkonst]

Could you explain why Example2_Parent() and Example3_parent() have different rendering behaviors even though they're the same code?

Children have got different hooks. React calls components by hooks. Example3 has got hooks, so React has got this capability to rerender children only. Hookstate is aware that parent has not used the state itself and just passed it to children. Hookstate is also aware whether a child has got the scoped state hook. Hookstate is also aware what segment of the nested state it used by every component. Having all this information, it triggers the rerendering for the smallest hooked possible part. In example 3 the smallest hooked would be children components. In example 2 the smallest hooked is only the parent.

[avkonst]

Would you then get

Example5_Parent   N   Y   Y
Example5_ChildA   Y   P   P
Example5_ChildB   N   Y   Y

Correct.

[avkonst]

If I set A, which of these should render?

I believe it will be only C and F.

[avkonst]

in the example above, I'd guess that only B, C, and F render when we set the value.

B would not rerender, because Hookstate traces that the component used the value and it is object, and it's A property has not been used. Setting A property of the object, does not change the object, it still remains to be an object with the same set of properties.

Currently, if you add a property to this object, then component B would rerender. for example: globalState.nested.C.set(5) . (Typescript would need to allow for this, if you declare property C to be an optional property of the state.). However, I have got the plans to optimise even this usecase, because it is possible to trace if Object.keys() has been invoked on an object or not. When this optimisation is done, component B would not rerender anymore, which would be also correct. In your example, you do not use any more information from the value except tracing the fact it is defined and of type object. So, even adding a property would not change the rendering result. But it is the future.

[avkonst]

However, this would trigger re-render:

function B() {
    const state = useStateLink(globalState)
        .with(Downgraded) // opts out from tracing usage of nested properties (improves the performance in some cases)
    const value = state.value // value is used, 
        // so it means Hookstate thinks that every nested property is used (when with the Downgraded plugin)
    return <p />
}

In the future, I am thinking to make Downgraded plugin globally enabled, and replace nested['property'] by pick('property') method. This would boost the performance of the library, because it would not need to use javascript proxies at all.

[avkonst]

Is this information helpful?

[jmchuster]

Thank you, this is all really helpful.

Ah, so your hint that "adding" a property by setting it to something for the first time counts as a change is exactly what I was experiencing. So now that behavior makes perfect sense to me.

I'm a little bit surprised that the following two act differently when changing the value of a property that already exists:

function B() {
    const state = useStateLink(globalState)
    const value = state.value
    return <p />
}
function C() {
    const state = useStateLink(globalState)
    const valueA = state.value.A
    return <p />
}

I understand that if you set a property on state that didn't previously exist, that "changes" it, so then the usage of state.value would cause it to re-render. However, if A already has been set but we're just changing the value of A, how do you know that function C uses state.A? I would have thought that once we deref it with state.value, that it's now a raw object beyond the domain of hookstate.

So then, does that mean that nested and value exist only so that we can create new scopes via useStateLink and have something we can call the set function on?

So for example, if I first create a stateLink

const globalState = createStateLink({
    days: [{
        jobs: [{
            startTime: new Date(),
            ...
        }, ...],
        ...
    },...] 
})

and then have the top line of every single one of my components as

const state = useStateLink(globalState).value

Then I should be able to navigate to whatever nested property I need, such as

const mondayFirstJobStartTime = state.days[0].jobs[0].startTime

and each copmonent will only re-render when one of its referenced values changes? So in this example, it would only re-render if the startTime property changed, or if the days array or jobs array was modified, or if a property was added or set on globalState or days[0] or jobs[0]?

[avkonst]

if A already has been set but we're just changing the value of A, how do you know that function C uses state.A?

Hookstate proxies the returned object, so it knows what properties you read from the object. And C function reads the property A in your example.

I would have thought that once we deref it with state.value, that it's now a raw object beyond the domain of hookstate.

It is the case if you apply Downgraded plugin as in my example above. However, if you do not apply the Downgraded plugin, this is not just raw object. It is an object which is watched by the Hookstate, and it knows how you use this object.

[avkonst]

So for example, if I first create a stateLink

const globalState = createStateLink({
    days: [{
        jobs: [{
            startTime: new Date(),
            ...
        }, ...],
        ...
    },...] 
})

and then have the top line of every single one of my components as

const state = useStateLink(globalState).value

Then I should be able to navigate to whatever nested property I need, such as

const mondayFirstJobStartTime = state.days[0].jobs[0].startTime

and each copmonent will only re-render when one of its referenced values changes?

Correct. Setting state.days[0].jobs[0].startTime, or state.days[0].jobs[0], or state.days[0].jobs, or state.days[0], or state.days, or state to a new value, will trigger rerender. Currently adding/deleting a property is also considered as setting the entire object.

So in this example, it would only re-render if the startTime property changed,

Correct.

or if the days array or jobs array was modified

If you set 0 day or zero job, then yes, correct. But if you set other days or jobs, then it would not rerender. If you add an element to days/jobs arrays then it will rerender (although quite few more optimisation are possible even in this case, but currently not available, maybe in the future).

, or if a property was added or set on globalState or days[0] or jobs[0]?

When new property is added to any of the parent objects, then yes it will rerender. If another (not in the path days.0.jobs.0.startTime) existing property is set, then it will not rerender.

[hyusetiawan]

I feel like this whole conversation should be a page in the guide under advanced. @jmchuster do you have the link to the blog post you're about to or have written about this?

[avkonst]

Yes. I do not close this ticket because plan to convert it to docs. The docs are under construction. I am just overloaded with work at the moment.

[jmchuster]

I'm still laying out how I want to order/sequence the posts. I'll post the drafts here for review once I have something.

[avkonst]

@jmchuster any update on the posts?

I have moved the most important parts of the conversation to the new documentation: https://hookstate.js.org/docs/performance-intro

[avkonst]

Let me know if you would like the chapter extended.