Sorting and pagination

[simenbrekken]

I've been experimenting replacing our own in-house redux data fetching implementation with redux-query. Fetching single entities or lists of those seem trivial, however I haven't been able to move beyond that so I was hoping you could answer a few questions regarding sorting and pagination.

Let's take a simple real-world example: You're fetching a list of orders from a REST API with three parameters: sortBy, limit and skip.

const OrdersContainer = connectRequest(({ sortBy = 'id', limit = 10, skip = 0}) => ({
    url: `/api/orders?sortBy=${sortBy}&limit=${limit}&skip=${skip}`,
    queryKey: `orders-sortBy:${sortBy}-limit:${limit + skip}`,
    update: {
        ordersById: (prevOrders, orders) => ({
          ...prevOrders, 
          ...orders,
        }),
    },
}))(OrdersContainer);

In the current version the queries reducer doesn't expose which entities the query resulted in, so I'm unable to know how many orders I've fetched using a given queryKey. Is there another way of doing this?

[RohovDmytro]

@simenbrekken I'm not sure, but if I tell you that you have access to sortBy, limit and skip values in your ordersById function, will it help? Or event better — use them in transform function.

[ryanashcraft]

@simenbrekken I'm a bit confused why you need to know how many orders you've fetched for a given queryKey. Can you explain at a higher level what the problem is? Is it just trying to get pagination working?

As @rogovdm suggested, typically we'd somehow reference the limit/skip/sortBy parameters in the update functions. I'd expect there to be some order ID array in your entities that is the source of truth for the ordering.

Also FWIW I rarely ever explicitly set the queryKey. Only time I've done it is to avoid possibly expensive serialization when there's large request bodies (typically for mutations or non-GET requests).

[simenbrekken]

@ryanashcraft @rogovdm I need to know how many entities have been fetched for a given query key to calculate how many items I need to fetch for the next page.

const OrdersContainer = connectRequest(({ sortBy = 'id', limit = 10, skip = 0}) => {
  const queryKey = `orders-sortBy:${sortBy}-limit:${limit + skip}`
  const available = state.queries[queryKey].result.length // I need this to calculate limit and skip

  return {
    url: `/api/orders?sortBy=${sortBy}&limit=${limit - available}&skip=${available}`,
   ...
  }
})(OrdersContainer);

[simenbrekken]

Arguably I should have spent some more time on this before I posted my question, but nevertheless here's my solution.

What this does:

• Loads a sorted list of orders from an API
• Normalize entities using normalizr
• Keep track of which entities were returned by what query. Orders sorted by createdAt and modifiedAt will sometimes point to the same underlying order entity but their order are completely different.
• Fetch as little data as possible using limit and skip

I've left out a few chunks for brevity.

const ListOrders = ({ orders, limit, fetchMore }) => {
  return (
    <div>
      {orders && (
        <div>
          <ul>
            {orders.map(order => (
              <li key={order.id}>#{order.id} - {order.items.length} item(s)</li>
            ))}
          </ul>
        </div>
      )}

      <button type="button" onClick={fetchMore}>Load more</button>
    </div>
  )
}

const Order = new schema.Entity('orders')

const getKey = JSON.stringify
const getOrdersByParams = ({ entities = {}, results = {} }, params) => {
  const resultKey = getKey(params)
  const orders = entities.orders
  const result = results[resultKey] || []

  return result.map(id => orders[id])
}

export default compose(
  connect((state, { sortBy }) => ({
    orders: getOrdersByParams(state.entities, { sortBy }),
  })),

  connectRequest(({ sortBy, orders, limit }) => {
    const available = orders ? orders.length : 0

    return {
      url: `/orders?sortBy=${sortBy}&limit=${limit - available}&skip=${available}`,
      queryKey: getKey({ url: '/orders', sortBy, limit }), // A limit: 10, skip: 10 is the same as a limit: 20, skip: 0

      transform: data => {
        const { entities, result } = normalize(data, [Order])
        const resultKey = getKey({ sortBy }) // Combine results that share parameters

        return {
          entities,
          results: {
            [resultKey]: result,
          },
        }
      },

      update: {
        entities: (prev, next) => merge({}, prev, next), // Poor man's entity management, won't remove entities that aren't present in next
        results: (prev = {}, next) => (
          Object.entries(next).reduce((result, [key, value]) => ({
            ...result,
            [key]: [...(prev[key] || []), ...value], // This problably needs to take limit and skip into account
          }), {})
        ),
      },
    }
  }),
)(ListOrders)

Hope this helps someone else.