Question - How to design state for a blogging application (that also works offline)?

[JeremyBernier]

I'm trying to create a simple blogging application, and am trying to wrap my head around how I would set up the state in Recoil.

Say we have a simple REST API /posts that returns a paginated list of posts, without the full post content of each post. The /post/:id endpoint returns a single full post including the content for a given post ID

What would be the optimal way to design the state in this application? To complicate things further, I'd also like for the state to be persisted in localStorage so that the application would work offline and be able to sync with the API later when back online, and also for caching purposes so we're not having to refetch data that's already been fetched (we can ignore this requirement for now though).

How should the state be designed in such an application? Really struggling to wrap my head around what should be an atom, atomFamily, selector, selectorFamily, etc. Also I'd love to see a GitHub repo or code sample of another recoil application of similar complexity including syncing with an API, since the client-side todo list in the tutorial is extremely trivial.

[tpatalas]

You can try atomEffect with IndexedDB. That is what I do.

[JeremyBernier]

Yea but how to design the atoms and selectors?

I don't see any sane way of doing this other than putting everything in a single atom, and then mutating via writable selectorFamilys that save to the API

[tpatalas]

Hope this helps you I do something like this with atomEffect. I basically created the reusable queryEffect as shown below.

export const atomQueryExample = atom<Example[]>({
  key: 'atomQueryExample',
  effects: [
    queryEffect({
      storeName: IDB_STORE['example'],
      queryKey: 'exampleID',
      cachedQueryFunction: () => getCachedData(CACHED_DATA['getData']),
      queryFunction: () => getDataExample({ model: SCHEMA_TYPE['example'] }),
      refetchOnMutation: true,
    }),
  ],
});

And these are the options I have created for queryEffect

    storeName // IndexedDB StoreName,
    queryKey // IndexedDB querykey,
    enableIndexedDb // boolean for IDB,
    queryFunction // api,
    refetchOnMutation 
    refetchDelayOnMutation ,
    refetchOnFocus,
    refetchOnBlur,
    refetchInterval,
    cachedQueryFunction // cache initial fetch to prevent multiple same requests,

I reuse the queryEffect to each atom/atomFamily, and the atomEffect lets you mutate/refetch/store into client-side storage such as localstorage or indexedDB.

This is the workflow of single atom with queryEffect:

1. useRecoilValue() somewhere to trigger initial fetch
2. fetched data will be store into indexedDB if it does not exist
3. if you update the state with setter function, it will update the indexedDB as well as remote database.
4. if you set the option true on refetch, it will refetch what you have just mutated and compare the data from remoteDB with indexedDB.
5. if data is stored into IDB, it will not trigger initial fetch, but it can refetch on mutation to refresh the IDB.
6. If you set the cacheQueryEffect, it will cache the request to prevent multiple requests. This is rare usecase because IDB is already prevent the multiple requests. I use this for prefetching the items before component mounted.

I think this is pretty much what you are trying to do except the offline to online update. I have not implemented it yet.

I don't think you really need to combine atom, selector, and so on. You can simply use atom with atomEffect. The difficult part is how you structure your re-useable effect I guess.

Hope this helps you build good idea.

[JeremyBernier]

Thanks I really appreciate your detailed help, and that makes sense.

Where I'm struggling with is how exactly to structure the atoms.

Recall that there are two REST API endpoints, /posts and /posts/id. Sure I could throw /posts in its own atom and /posts/id in its own atom family, but then we've got duplicate overlapping data from those two calls.

It makes sense to have a cache that holds a map of the post ID to the post content, along with an array of the posts in their ordering. I just don't know how to accomplish this in any other way than having all of that data combined inside a single atom.

I can see that Recoil effects can make an atom automatically save to localStorage or an API on change via onSet. However it doesn't seem possible to fetch from an API when an atom has already been initialized. So if I put all my state in a single atom and then later I request data from it that it doesn't have (eg. because /posts is paginated or only returns partial data when I'm looking for the full content of a post /posts/id), then there's no way for me to define in any recoil effect to automatically fetch from the API. The only way to accomplish this would be outside of Recoil such as by creating a custom React hook replacement for useRecoilState that checks if that recoil atomFamily for a given post ID is populated and if it isn't then fetches from the /posts/id API and then updates the atomFamily. This seems pretty pretty hacky though and not how recoil was intended to be used (not to mention having to manually track loading/error states, something libraries like react-query handle outside of the box).

[tpatalas]

It makes sense to have a cache that holds a map of the post ID to the post content, along with an array of the posts in their ordering. I just don't know how to accomplish this in any other way than having all of that data combined inside a single atom.

aw I miss understood some parts. I agree. In your case, I would avoid putting a chunk of data into a single atom. Instead of fetching the array of data, I will structure the query in the backend to fetch only the list of the ids in an array object you desire to fetch. Then you can fetch each small post set with atomFamily by passing the Id you have just fetched. The drawback is that you will fetch multiple items in parallel, which looks not clean in the network tabs. So it will be structured as 1) fetch the list of ids and 2) fetch each item separately. If you want to speed up the process, useRecoilCallback to prefetch the ids ahead of time. That is what I do.

I can see that Recoil effects can make an atom automatically save to localStorage or an API on change via onSet. However it doesn't seem possible to fetch from an API when an atom has already been initialized.

I do not quite understand this. When you structure your atomEffect, You can create separate query functions to fetch for onSet and trigger === get along with setSelf. You can trigger however you want to refetch after the atom is initialized with the initial fetch. Recall we assume to structure the query data with ids only initially then fetch each item? If you trigger the refetch of those ids, your atomFamily (child atom I should say) will automatically create the atom with new data. For example, You have 3 posts; then you create another post outside your application, such as from a database. The only thing you need to do is refetch those ids. It will create a new atom without even triggering the suspense. I have tested this quite often, so I am glad I can clearly say this. Therefore, now you have 4 posts. Although Recoil uses cache itself, if we even use IDB in this case, It will be even better.

So if I put all my state in a single atom and then later I request data from it that it doesn't have (eg. because /posts is paginated or only returns partial data when I'm looking for the full content of a post /posts/id), then there's no way for me to define in any recoil effect to automatically fetch from the API.

You are right. If you decide to fetch at once instead of the method I have mentioned above, You don't need atomEffect on the atomFamily. You can just fetch with a single atom and then pass the data to atomFamily with an array map (simpler). Although this is okay, updating and refetching parts become a little messy because you rely on a single fetch. You can also use the selector to fetch, but It is Read-Only, so it gets trickier and inefficient when you want to refresh the data, such as hammering a whole cache to trigger the refetch. I would always prefer to use bi-directional sync such as atomEffect.

The only way to accomplish this would be outside of Recoil such as by creating a custom React hook replacement for useRecoilState that checks if that recoil atomFamily for a given post ID is populated and if it isn't then fetches from the /posts/id API and then updates the atomFamily.

That is overthinking. I would fetch ids first, then fetch items. I guess there are many different ways to structure.

libraries like react-query handle outside of the box

Everyone loves react-query, including myself.

Hope this adds some value to you

I think Next.js's server-side rendering is also good option especially for blogging. Not sure about offline mode though.

[JeremyBernier]

I appreciate your help! I just switched to Zustand because it's way easier. Highly recommend that to anyone else reading this who also finds Recoil extremely confusing.