blog/react-query-data-transformations

[utterances-bot]

React Query Data Transformations | TkDodo's blog

Learn the possibilities to perform the quite common and important task of transforming your data with react-query

https://tkdodo.eu/blog/react-query-data-transformations

[hrafnkellpalsson]

The select is nifty! Haven't tried out v3 yet, looking forward to trying out.

As a slight variation on 1 I've sometimes put both the untouched server data and the derived data into the cache, something like

const fetchTodos = async (): Promise<Todos> => {
    const response = await axios.get('todos')
    const data: Todos = response.data
      return {
        server: data,
        derived: data.map((todo) => todo.name.toUpperCase())
      }
}

[TkDodo]

That looks good if you need both the raw data and the derived data in the cache 👍

[bebiangel]

Wow!! That's really good.👍 Thanks Dominik!🙏

[onkarj422]

Hello, thanks for sharing these useful insights!

What different does it make if the select function is an inline one vs a stable function reference?

[TkDodo]

What different does it make if the select function is an inline one vs a stable function reference?

if it's an inline function, it will be executed on every render, even if neither data nor any other input to the function has changed, because the identity of the function itself has changed. This is only relevant if your component renders for another reason, and also only if your transformation is very expensive.

I suggest to just start with an inline function, and if you measure some slowness, you can improve it. It's likely perfectly fine to inline it :)

[jacobworrel]

Thank you for this great article series!

Correct me if I'm wrong, but since the select option runs once per component instance, what would you recommend if you only want a data transformation to run once for multiple component instances? For example, let's say we have a useQuery call with a select option that runs an expensive computation wrapped in a custom hook and that custom hook is called by many components throughout an application. Is the only option then to move the data transformation to the queryFn and have it run once per fetch?

[TkDodo]

If the select function is stable, it will only execute when data changes. So even if you have it in multiple components, it will not run when re-rendering, and even if you have a background refetch, it will not run unless the data is different (after structural sharing). So yeah I'd do that first, because moving it to the queryFn means it will run with every request, which might be more often.

[jacobworrel]

Thanks for the clarification! I made a codesandbox to better illustrate the example I'm talking about where the select function is called once per component that's consuming the custom hook with the useQuery call, despite having a stable function reference (I think). What am I missing?

[TkDodo]

You’re right. select is a per-observer option, so every usage will have it's own select memory, which means it has to at least run once per observer.

It seems that indeed the only other option is to move it into the queryFn. I don’t think there is a way to transform there only once, but something like memoize-one could help.

[wobsoriano]

I'm having problems with useInfiniteQuery and select option.

export function useConversation(to: string) {
  const user = useUser();

  return useInfiniteQuery<MessageListResponse>(
    [QueryKeys.MessageList, { userId: user?.guid, to }],
    ({ pageParam = 1 }) => {
      return $axios.$post(`/api/${QueryKeys.MessageList}`, {
        guid: user.value?.guid,
        to: to,
        offset: pageParam
      });
    },
    {
      getNextPageParam,
      enabled: !!to,
      select: (data) => { // type error
        const pages = data.pages.map((page) => {
          return  transformList(page.payload.list);
        });

        return {
          ...data,
          pages,
        };
      },
    }
  );
}

Without the select property, the type I'm getting is

undefined | InfiniteData<MessageListResponse>

But with the select option, I'm getting type errors.

[TkDodo]

@wobsoriano I can take a closer look at it if you can create a minimal example in codesandbox

[wobsoriano]

@wobsoriano I can take a closer look at it if you can create a minimal example in codesandbox

Here you go https://codesandbox.io/s/use-query-infinite-ts-583sp

[TkDodo]

Ah yes, you cannot use select to change the whole structure with an InfiniteQuery. You need to still return an object with pages and pageParams (known as InfiniteData). You can structure the pages however you like though. See also this discussion: https://github.com/tannerlinsley/react-query/discussions/1410

[wobsoriano]

Ah yes, you cannot use select to change the whole structure with an InfiniteQuery. You need to still return an object with pages and pageParams (known as InfiniteData). You can structure the pages however you like though. See also this discussion: tannerlinsley/react-query#1410

Ah, that makes sense. I updated my example and used your 2nd option instead. Works fine.

[phutngo]

Thank you so much for this guide! I find myself keep coming back here as a reference. A couple of months back, we chose to use the "select" method to do our data transformations and it's been working out great. It seems like it's the most flexible pattern, and the code is easy to read and understand as well.

[MHebes]

Great writeup and blog, thank you!

You’re right. select is a per-observer option, so every usage will have it's own select memory, which means it has to at least run once per observer.

I'm a bit confused how this works if you don't mind explaining.

Are all options per-observer, and only the queryFunction is deduplicated amongst the same queryKey?

In other words, is it true that two useQuery calls, with the same key in separate components, should always use the same queryFn, but may safely have any options they want?

[TkDodo]

@MHebes some options work per observer (like select), but some also work on the cached entry itself. Notable examples would be initialData (there can only be one initialData, because it goes directly to the cache) or staleTime (because there is only one query that can become stale).

The queryFn also doesn't make much sense to be on a per observer level, as it fills the same cache entry. So it should really be the same, unless you have two endpoints that deliver the same structure of data. But even then, I would probably use a separate useQuery :)

[rnnyrk]

What if you want to combine useQuery for an output?

For example (bit simplified ofc):

export const useListTasks = (category: i.Categories): UseQueryResult<i.Task[] | undefined> => {
  return useQuery(
    category ? ['tasks', category] : 'tasks',
    async () => await fetcher(LIST_TASKS, { category }),
  );
};

export const useQueryUserTasks = ({ id?: string, category: i.Categories }): UseQueryResult<i.UserTask[] | undefined> => {
  return useQuery(
    ['userTasks', id, category],
    async () => await fetcher(LIST_TASKS, { category }),
    {
       enabled: Boolean(id),
    }, 
  );
};

export const useSelectUserTasksByType = (category: i.Categories): i.UserTasksByType => {
  const { data: tasks } = useListTasks(category);
  const user = useQueryUser();
  const { data: userTasks } = useQueryUserTasks({ id: user?.id, category });

  return React.useMemo(() => {
     // .... do data formatting
  }, [tasks, userTasks]);
};

Is this considered good practice to combine data queries?

[TkDodo]

if you have two independent queries and need to combine their data then yes, that will work nicely. There might be some small re-rendering improvements if you'd use select on the second query and closure over the result of the first query, because useMemo can't bail out of re-renders, but it's very likely not important

[ivan-kleshnin]

Great article Dominik! Never heard of selectors previously, thanks for bringing them up.

I agree with your points but I think one BIG downside of putting data transformations in hooks, that wasn't mentioned, is that it's impossible to reuse hooks in SSR (think NextJS getServerSideProps). And you don't want to repeat logic across SSR handlers and components.

Our teams keeps the majority of data transformations in custom fetchers like fetchArticle, fetchTalent, etc.

But having N custom fetchers + N custom hooks is too much of a boilerplate. So we decided to avoid custom hooks and we use just one custom wrapper around useQuery which relies on unified API across fetchers (all fetchers accept params and query) and exposing key (prefix). Smth. like:

export type QueryFn<P, Q, R> = {
  (params : P, query : Q) : Promise<R>
  key : string
}

export function useQuery<P, Q, R>(
  queryFn : QueryFn<P, Q, R>,
  params : P,
  query : Q,
  options ?: ReactQuery.UseQueryOptions<R, api.ApiError, R>
) : ReactQuery.UseQueryResult<R, api.ApiError> {
  const queryKey = [queryFn.key, params, query]
  return ReactQuery.useQuery<R, api.ApiError>(queryKey, () => {
    return queryFn(params, query)
  }, options)
}

Then we can use simple useQuery(api.fetchArticle, {articleId: ...}) in components and api.fetchArticle in SSR. useMutation and useInfiniteQuery are wrapped similarly. I'm not 100% sure that hardcoding keys like that won't blow up in the future, but it works so far.

[TkDodo]

I agree with your points but I think one BIG downside of putting data transformations in hooks, that wasn't mentioned, is that it's impossible to reuse hooks in SSR (think NextJS getServerSideProps). And you don't want to repeat logic across SSR handlers and components.

That's true, thanks for bringing it up. select doesn't suffer from that because you'd just put the raw data into the cache and have each observer pick the slice they are interested in with select. I think it's overall the best of the options :)

[lintonye]

Great article!

One question though. I'm not sure how useCallback would help memoize an expensive transformation. It memoize the function itself but not its execution, right? Did I miss anything?

Thanks.

[TkDodo]

One question though. I'm not sure how useCallback would help memoize an expensive transformation. It memoize the function itself but not its execution, right? Did I miss anything?

@lintonye react-query will invoke the select function on any render if:

• data changed, because that's the input for select
• the reference to the select function itself changes

So if you make the select function stable, either by extracting it to a stable function reference outside of your component, or by memoizing it with useCallback, it will not be executed by react-query unless necessary.

[bsides]

Does anyone know what is the type of select? I'm using this by inference, but it's very verbose and not very generic:

type SomeData = {} // data returned from api
type ReactQuerySelectType =
  | ((data: SomeData[] | undefined) => SomeData[] | undefined)
  | undefined

EDIT: well, to make it more generic, one could just use like this:

type SomeDataType = {} // data returned from api
type ReactQuerySelectType<T> =
  | ((data: T[] | undefined) => T[] | undefined)
  | undefined

function someFunc(select: ReactQuerySelectType<SomeDataType>) {
  return useQuery(['someQuery', fetcher, { select })
}

Although it serves its purposes, if anyone know a better way let me know.

[TkDodo]

@bsides I usually do this:

const useSomeFunc = <T = SomeDataType>(select?: (data: SomeDataType) => T) =>
  useQuery(['someQuery', fetcher, { select })

the default value on the generic is necessary so that you get the correct inference even if you don't provide a select function. If the select param to your hook is mandatory, you can omit it.

[bsides]

Nice one, I'll try that! Thanks again @TkDodo

[woohm402]

Great article 🙌 thanks!

I have a stupid question: what do you think is the best practice for infinite query? Using the awesome select option works great with useQuery because it has a single data, so it is clear that we have to run select callback with the single data every time when the fetched data has changed.

However useInfiniteQuery has multiple data (page): select callback receives entire pages and returns entire pages. It seems inefficient because in most cases, we would have already done data transformation for previous pages.

example snippet

Assume that `queryFn` returns `Promise` with size 100 ```js const { data } = useInfiniteQuery(key, fn, { select: ({ pages, pageParam }) => ({ pages: pages.map( (page) => page.map( (item) => item ** 2 // point on this line ) ), pageParam }) }) ``` With the snippet above, `point` will run 100 * (page count) times. If page count is 10, the line will run 1000 times even if first 900 executions are unnecessary.

My words were too long :sweat_smile: Returning to the question, what is your opinion for data transformation with infinite query? I believe that my snippet above will cause slow render.

[TkDodo]

what is your opinion for data transformation with infinite query? I believe that my snippet above will cause slow render.

I don't think that iterating over one array 1000 times will get you in any significant trouble, but yes, select is a bit awkward with useInfiniteQuery because it requires you to return the same structure with pages and pageParams. For example, right now, you can't select the count of all posts with an infinite query and select. It works at runtime, but there's a problem with typings. I've tried a couple of times to fix it, see here: https://github.com/tannerlinsley/react-query/issues/3065#issuecomment-996561089

So yeah, when it comes to useInfiniteQuery, I've mostly done no transformations on the frontend at all, or I went the useMemo route.

[LautaroRiveiro]

Could it be a situation where you have to pay attention using method 1 (queryFn) with queryCache? For example, you update some data and the service returns the new data in its own format. Should you also transform it here before updating the cache manually? Sorry, I am new. Thanks for the post.

[TkDodo]

@LautaroRiveiro the data in the cache should always be in the same format for one key. So if you update the cache manually, it has to be in the same format as the one that the queryFn returns.

[woohm402]

So yeah, when it comes to useInfiniteQuery, I've mostly done no transformations on the frontend at all, or I went the useMemo route.

Thanks @TkDodo !

[coltanium13]

@TKDodo, whenever i write a select for the useQuery option, why does the react-query-dev-tools not show anything in the Query Explorer > options > select? Am i doing it wrong, or does it not work?

Thank you for everything, your blog is extremely helpful!

[TkDodo]

@coltanium13 select is a function, which cannot be visualized in the devtools. We also don't show the result of the select function in the devtools, because the devtools only show what is in the cache. For each cache entry, you can have many observers, each one potentially selecting their own slice of data.

[halbano]

How can I post process data for entry of results using useQueries?

results being:

const results = useQueries(queries);

Meanwhile the queries are being returned, I would like to keep a collection of the post processed items returned by the parallel queries

[TkDodo]

to transform each item individually, you can still use select on each query. For a transformation on the whole result, you'd need something like transform(results.map(result => result.data)). With the new useQueries api in v4, we now have the option to add a top level select at some point in the future.