Complex Params with Singleton Atom

@bowheart Thank you very much this wonderful library. At least my first impressions are that it's amazing. I'm in the process of learning it, but I think I have a pretty good overview.

I had a question, I was hoping you would help me with it.

I noticed that the atom state factory "creates a different atom instance for every 'unique' set of params you pass" and unlike React, "Zedux doesn't compare object references. Internally, Zedux turns all params into a single string."

Imagine, however, if I pass a huge list where each item in the list is a deeply nested object with many, many properties, and I want to use this as the initial state of the injected atom's store. This doesn't seem to bode well however with how the atom state factory works.

It only needs to be a singleton store/atom.

How would you recommend that I solve this? I have considered the following solutions, but IDK exactly:

Setting complexParams: true when creating the ecosystem. The example you gave was just for a function as argument, but perhaps this would also work for arrays/objects?
Creating an atom where the valueOrFactory is the store. But there are all sorts of problems that come with this.
Perhaps somehow creating the store outside the atom, and then passing the store into the atom state factory as its parameter? But according to your docs the args of the atom state factory must be serializable, and I would imagine that stores are not serializable?

Theoretically speaking, all I think I need for this case is a sort of singleton atom with a predefined store. That store needs to change, but that atom should never be recreated if the original params to it somehow change.

I can try to give more detail, or perhaps you understand where I'm coming from? It seems like this would be a sort of common use case, but I'm not sure there exists a great solution for it yet?

Pretty sure that if you pass a function as a parameter you’ll see the part of the key it generates is just undefined or null - can’t remember exactly off top of my head. So if your function changes, Zedux won’t treat it as a new parameter and won’t trigger a new instance. Probably because at some point JSON.strongly is called on the parameters to make the keys.

So, that’s a long winded way of saying you can probably pass in a function that your atom can call to create its initial state for a store.

On Sat, Nov 2, 2024 at 11:41 AM Nick DeCoursin @.***> wrote:

@bowheart https://github.com/bowheart Thank you very much this wonderful library. At least my first impressions are that it's amazing. I'm in the process of learning it, but I think I have a pretty good overview.

I had a question, I was hoping you would help me with it.

I noticed that the atom state factory https://omnistac.github.io/zedux/docs/walkthrough/atom-instances#atom-params "creates a different atom instance for every 'unique' set of params you pass" and unlike React, "Zedux doesn't compare object references. Internally, Zedux turns all params into a single string."

Imagine, however, if I pass a huge list where each item in the list is a deeply nested object with many, many properties, and I want to use this as the initial state of the injected atom's store. This doesn't seem to bode well however with how the atom state factory works.

It only needs to be a singleton store/atom.

How would you recommend that I solve this? I have considered the following solutions, but IDK exactly:

Setting complexParams: true when creating the ecosystem. The example https://omnistac.github.io/zedux/docs/advanced/complex-params#complexparams you gave was just for a function as argument, but perhaps this would also work for arrays/objects?

Creating an atom where the valueOrFactory https://omnistac.github.io/zedux/docs/api/factories/atom#valueorfactory is the store. But there are all sorts of problems that come with this.

Perhaps somehow creating the store outside the atom, and then passing the store into the atom state factory as its parameter? But according to your docs the args of the atom state factory must be serializable, and I would imagine that stores are not serializable?

Theoretically speaking, all I think I need for this case is a sort of singleton atom with a predefined store. That store needs to change, but that atom should never be recreated if the original params to it somehow change.

I can try to give more detail, or perhaps you understand where I'm coming from? It seems like this would be a sort of common use case, but I'm not sure there exists a great solution for it yet?

— Reply to this email directly, view it on GitHub https://github.com/Omnistac/zedux/issues/127, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABBKOGN7T47FKGRPFC7635TZ6TXCPAVCNFSM6AAAAABRB2RUCCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGYZTANJTG4ZTANQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

Hi @ChrisBoon that's an interesting suggestion, but that approach is definitely a little hacky and would be confusing to people reading the code.

If the complexParams on the ecosystem were later changed to true, that would cause the singleton atom to no longer be singleton if invoked in multiple places, or if someone forgot and change it to true it would break my app. And that would force lock-in to always having complexParams: false.

I really think there should be a way to pass in other data to the atom state factory that is not included in the "hashing". Is there some reason that hasn't been done yet, or perhaps just oversight?

Hey @decoursin great question. And that's a cool suggestion @ChrisBoon. I'm gonna play with that actually.

Firstly, if you're running into this frequently, you're probably underutilizing Zedux. In an ideal setup, that big nested list would already be controlled by Zedux - e.g. it would be fetched/created in another atom rather than in a React component. The new atom could then simply access the list via an injector or atom getter. This setup also allows the new atom to react to changes in the list rather than taking a static snapshot on init.

With that in mind, there are exceptions, especially when using 3rd party tools that rely on React hooks. So I'll assume that's the case here.

complexParams isn't what you're looking for. Since the atom is for sure a singleton, you want it to have no params.

The Approaches

There are a few ways to go about it. They all basically come back to one idea:

Get that data in the ecosystem somehow.

Use another atom

const externallyPopulatedAtom = atom('externallyPopulated', null)

const bigListAtom = ion('bigList', ({ get }) => {
  const initialState = get(externallyPopulatedAtom)
  const store = injectStore(initialState)
  ...
})

function SomeTopLevelComponent() {
  const ecosystem = useEcosystem()

  useBigListFetcher(
    bigList => ecosystem.getInstance(externallyPopulatedAtom).setState(bigList)
  )
  ...
}

Set state after init

const bigListAtom = atom('bigList', () => {
  const isInitializedRef = injectRef(false)
  const store = injectStore([])

  return api(store).setExports({
    init: initialState => {
      store.setState(initialState)
      isInitializedRef.current = true
    }
  })
})

function SomeTopLevelComponent() {
  const bigListInstance = useAtomInstance(bigListAtom)

  useBigListFetcher(bigList => bigListInstance.exports.init(bigList))
  ...
}

Note that this can usually be even simpler. I'll put that example towards the end.

Use ecosystem context

Ecosystems have an optional .context property that can hold anything. I don't really recommend this approach since it isn't type safe, but it can be a useful escape hatch sometimes.

// With an ecosystem created like this:
const ecosystem = createEcosystem({
  context: { bigList: null },
  id: 'root',
})

// you can mutate and read from `ecosystem.context` at will
const bigListAtom = ion('bigList', ({ ecosystem }) => {
  const store = injectStore(ecosystem.context.bigList)
  ...
})

function SomeTopLevelComponent() {
  useBigListFetcher(bigList => {
    ecosystem.context.bigList = bigList
  })
}

This could also work with your store suggestion - create a store outside the atom with any state, put the store on the context, and grab it in bigListAtom.

If you do find this approach useful, let me know. I'm considering potentially deprecating ecosystem context and removing it in the future.

Note on Timing and Errors

None of these examples included a way to ensure that normal consumers of bigListAtom only use it after the list has been populated. There are several ways to go about it. Some examples:

Suspend in a top-level component until the list is populated.
Attach a promise directly to bigListAtom to suspend all components that use it.
Throw an error in bigListAtom if any consumers try using it before the list is populated to alert yourself of new timing problems that need manual adjusting.
Allow bigList to be empty and handle that default in all consumers.
Force consumers to access bigListAtom through an atom selector.

We pretty much always use the last two options.

Handle empty value

This is the most common way to deal with timing issues - make consumers handle an empty default. They'll reactively update when the data does come in anyway. This is usually very easy with arrays since consumers can handle an empty array exactly the same as a non-empty array.

So here's a full example of the most common way to initiate an atom with complex external data:

const bigListAtom = atom('bigList', [])

function SomeTopLevelComponent() {
  const bigListInstance = useAtomInstance(bigListAtom)

  useBigListFetcher(bigList => bigListInstance.setState(bigList))
}

function SomeConsumerComponent() {
  const bigList = useAtomValue(bigListAtom)

  // works whether the list has been populated yet or not:
  return bigList.map(item => <SomeItemComponent key={item.id} item={item} />)
}

Usually this is all you need.

Access via a selector

Consider this a postscript. This is an advanced pattern and may be overkill for you. If your question is already answered, ignore this. That said, we've started using this pattern in a few places and really like it. The gist:

Force all consumers to access the atom via an atom selector
Since selectors are plain functions, you can call them directly with ecosystem._evaluationStack.atomGetters as the first arg. Do that to initialize the atom.
All other consumers use select/similar without passing the initial state

// don't export the atom:
const bigListAtom = atom('bigList', ...)

// only export this atom selector:
export const getBigListInstance = ({ ecosystem, getInstance }: AtomGetters, initialState?: BigList) => {
  if (!initialState) {
    // make sure the instance already exists so the `getInstance` call doesn't create it
    if (!ecosystem.find(bigListAtom)) throw new Error("can't access bigList before init")

    return getInstance(bigListAtom)
  }

  // If it hasn't been initialized, initialize it
  const instance = getInstance(bigListAtom)
  instance.exports.init(initialState)
  return instance
}

// then initialize the instance like so:
getBigListInstance(ecosystem._evaluationStack.atomGetters, bigList)

// and all other consumers access it without passing the arg. E.g. in React components:
useAtomSelector(getBigListInstance) // to get the instance
useAtomValue(useAtomSelector(getBigListInstance)) // to subscribe to the value

@bowheart thank you very much for your very wonderful detailed description. It introduces some new ideas for me, and confirms others I wasn't totally certain about.

Firstly, if you're running into this frequently, you're probably underutilizing Zedux. In an ideal setup, that big nested list would already be controlled by Zedux - e.g. it would be fetched/created in another atom rather than in a React component. The new atom could then simply access the list via an injector or atom getter. This setup also allows the new atom to react to changes in the list rather than taking a static snapshot on init.

I'm actually using SSR with a very popular react meta framework. I'm fetching the data first on the server side, then passing it down through the client side as props. This is a common pattern that I'll be repeating throughout my app. For a lot of reasons, it's better to request the data immediately on the server side, rather than waiting for the client to request it inside an atom.

I realize that Zedux supports SSR itself to a certain extent. Therefore, theoretically, I would maybe be able to instantiate the atom on the server side, then pass it down to the client somehow using Zedux' SSR capabilities by dehydrating and then rehydrating the ecosystem, but I don't think that's a very good/clean solution especial the ease and convenience of just passing the data down to the client as props.

Your two suggestions "Use another atom" and "Set state after init" are nice, but they come with a ton of other problems which you outlined fabulously yourself. Actually, your "Use ecosystem context" suggestion would maybe be the best approach, but it doesn't seem that stable according to you (ha) and for other reasons like issues with type safety like you suggested and probably more difficulty debugging, etc.

I'm so far in love with this library. But is there any functional reason why you don't want to add someway for users to pass additional data to the atom that isn't part of the hashing? I realize it would maybe require a large refactor or something, but it seems to me like it would be something feasible and have many uses cases.

@decoursin Ah I was approaching this from a completely different angle. With SSR, it sounds like Zedux's hydration is what you want then. You don't have to pair it with dehydration on the server - it sounds like your framework already does that.

Exactly how that looks depends on your setup. Without knowing that this is kind of a shot in the dark, but I know that in Next 15, you can call ecosystem.hydrate with retroactive: false directly in a client component function body. Just specify the atom key(s) you want to hydrate directly:

'use client'

const bigListAtom = atom('bigList', [])

function MyClientComponent({ bigList }) {
  const ecosystem = useEcosystem()

  // hydrating right here may be fine depending on setup:
  ecosystem.hydrate(
    { [bigListAtom.key]: bigList },
    { retroactive: false }
  )

  // now access bigListAtom
  const listFromAtom = useAtomValue(bigListAtom)

  return <div>Hydrated {listFromAtom.length} values</div>
}

Actually, your "Use ecosystem context" suggestion would maybe be the best approach

It's just that it's unnecessary. If you're mutating ecosystem context, you have access to the ecosystem and can be updating the state of an atom instead:

ecosystem.context.bigList = bigList // not type-safe, not trackable, not reactive
ecosystem.getInstance(bigListAtom).setState(bigList) // type-safe, trackable, reactive

I've tried to setup the SSR using dehydration/hydration, and it gets really complex fast. It's definitely not the direction that would make sense to go.

I would imagine you haven't done this before, because if you have, you know how complex it gets.

For starters, moving my http request into an atom results in a Query Atom. Query Atom cause additional complexities.

Query Atoms don't return pure data, they return PromiseState, which isn't actually a promise. PromiseState complies with the React's Suspense, but you can't await on it or perform other promise-like operations.

Now imagine you have a Query Atom on the server that you want to await on (because other data/queries/computation depend on it before you respond to the client request). How can you do it? As far as I can see, you can't (because you can't await on a PromiseState). If you want to, everything has to become part of the Zedux Atom universe which is no-go from the start. At this point Zedux would no longer be a library, but a "framework" where everything has to be done in the framework otherwise it's not possible.

Query Atoms return PromiseState, PromiseState complies with Suspense, but Server Components don't work with Suspense. Suspense is fundamentally meant to provide React with the capabilities of running concurrently inside Browsers - it's not meant for Server Components.

Besides the problems causes by Query Atoms, you have all the hydration/dehydration code that becomes quite a mess. Hydrating on the server, dehydrating on the client, what should be "retroactive" what shouldn't. Performance penalties of hydration? Are my atoms serializable? Debugging issues. etc.

I'm not trying to bash Zedux. Once again, I find Zedux to be amazing. But I personally think Zedux is a paradigm meant for client code / Client Components, it's not meant for server code / Server Components. The entire reactivity paradigm with "molecules" or "atoms" is a frontend/client paradigm, not a server paradigm. Once you start using Atoms on the server, half of the stuff doesn't even work anymore (at least for react it's so) for good reason, like useEffect, useMemo (etc) none of these things work or make sense on the server and likewise I'm assuming injectEffect, injectMemo (etc) don't work or make sense either.

Furthermore, once you have a codebase mixing server-side atoms with client-side atoms things start to become really complex and hard to grasp. Spaghetti code. It's like mixing Christmas in the Winter and Christmas in the Summer - it doesn't make sense / it's not consistent.

All of this in comparison to just running my http request inline in TS then passing the data down to the clients as props. This is obviously far far easier and more maintainable. The other solutions that you offered are definitely better in my opinion than trying to do SSR with atoms.

TL;DR

Only use Zedux in client components. If there's a demand for using Zedux in RSCs, it's possible but we need better docs.

Full Answer

imagine you have a Query Atom on the server that you want to await on (because other data/queries/computation depend on it before you respond to the client request). How can you do it?

@decoursin Query atoms set a .promise on the atom instance you can await in an RSC like so:

'use server'

export async function MyServerComponent() {
  const queryInstance = serverEcosystem.getInstance(myQueryAtom)
  await queryInstance.promise

  return <MyClientComponent data={queryInstance.getState().data} />
}

But before we get too caught up in that approach, we should come back to this:

Zedux on the client

Zedux is a paradigm meant for client code / Client Components, it's not meant for server code / Server Components

Yes! Exactly this. The mess you're seeing is due to how frameworks (especially nowadays with RSCs) work. It isn't specific to Zedux - it would be the same with any 3rd party state manager. This complexity is why most people using frameworks avoid 3rd party state managers completely.

That may be my advice for you too. If Zedux isn't solving a problem or making things easier for you, you probably don't need it. That said, I do use Zedux in Next. Let me reiterate this:

You don't have to pair it with dehydration on the server - it sounds like your framework already does that.

Contrary to what I said before when I was assuming a client-only setup (which is where Zedux and all 3rd party state managers shine), it now sounds to me like you're trying to fight the framework by overutilizing Zedux. Your framework already has RSC support. I'd use that, not query atoms in Zedux.

I believe frameworks could be much more powerful if they integrated more deeply with 3rd party state managers. Unfortunately they don't. That means lots of manual work if you want to use Zedux (or any similar lib) on both the server and the client.

The fix is simple in theory, though will take a while to get used to (which is just RSCs in a nutshell and again isn't Zedux's fault, it's the same for any 3rd party state manager): Only use 3rd party state managers on the client.

I really only know Next. I'll try to make some time to experiment with Remix and Waku. But in Next at least, the server counterpart to my hydration example is simply:

'use server'

export async function MyServerComponent() {
  const bigList = await fetchBigList() // no Zedux involved

  return <MyClientComponent bigList={bigList} />
}

If you don't want to worry about retroactive: false, you can create a custom hook like this that covers all cases:

const useAtomHydration = <T extends AnyAtomTemplate>(
  atomTemplate: T,
  val: AtomStateType<T>
) => {
  const ecosystem = useEcosystem()
  const existingAtom = ecosystem.find(atomTemplate)

  // hydrate immediately if the atom doesn't exist yet (ideal)
  if (!existingAtom) {
    ecosystem.hydrate({ [atomTemplate.key]: val })
  }

  // defer to a useEffect to set ("hydrate") the existing atom's state safely
  useEffect(() => {
    if (existingAtom) existingAtom.setState(val)
  }, [val]) // don't add `existingAtom` - it isn't reactive (and shouldn't be)
}

Then use like so:

function MyClientComponent({ bigList }) {
  useAtomHydration(bigListAtom, bigList)
}

Zedux on the Server

While I'd avoid using Zedux on the server in general, its advanced cache management does make it a suitable replacement for Next's unstable_cache and React's cache function. Atoms can also be flagged as either global or temporal/route-specific and their lifecycles can be controlled per-RSC.

This requires quite a bit of setup. Depending on the framework/compiler, there may be manual work you'd have to remember to do per-RSC too. But at least in Next when not using Turbopack, TypeScript's using keyword is amazing to prevent manual cleanup. I'm not sure we have enough documentation for me to encourage it, but it's possible.

If people are interested in going down this route, then we should improve the docs.

Some of your confusion is probably from the SSR guide in the docs. That's a low-level guide for doing SSR from scratch with no framework. It's probably misleading. We should bury that guide more and replace that with a guide for people using frameworks since that's way more common.

Additionally, since frameworks are so common in general, we should probably add notes throughout many doc pages of APIs that are only designed for use on the client.

Hi @bowheart thank you again for your detailed description. I see that I made a couple mistakes, but I also see that we agree on a lot of what I said - so that's good:)

I will have to play around with your useAtomHydration function, which I'll definitely get to later this week. Haha but my first impression still is that it's still more complex than if it were possible to simply pass in the initial state as an argument into the atom state factory function.

I really appreciate all the time you've given me on this.

I've looked into all the options, and for the problem at hand I'll actually be using @ChrisBoon's solution. Thank you very much Chris! @bowheart thanks again for the wonderful library. I hope to contribute back in the future!

Omnistac / zedux