DataJunction / dj

A metrics platform.
http://datajunction.io
MIT License
35 stars 15 forks source link

Pagination #1203

Closed shangyian closed 4 weeks ago

shangyian commented 1 month ago

Summary

Retrieving nodes for displaying on the homepage can be quite slow if there are many nodes. This PR adds pagination to the findNodes query in GraphQL, and switches the UI to use the GQL endpoint instead of REST.

Backend

This change adds a new findNodesPaginated query to allow for more gradual migration from the existing findNodes query. We implement cursor pagination, based on the GraphQL Relay spec. The encoded cursor is of the node's (created_at, id) tuple. The setup supports both forward and backwards pagination with startCursor and endCursor.

Also added indexes on relevant fields, like node's created_at and namespace, to make it more likely that database queries, especially cursor-based queries, will use indexes.

UI

This change switches the UI to depend on the GraphQL server, using the findNodesPaginated query to build a limited set of nodes for the landing page. Additional nodes can be retrieved through the Next and Previous buttons:

Screenshot 2024-10-21 at 10 57 50 PM

There's another change that will make a query every time the filter selection at the top of the page changes. This is because the original implementation would always load all nodes for the selected namespace, and without pagination, this could take a long time. However, once this list of nodes was loaded, filtering was just a matter of client-side filtering in React. With the new setup, we have to re-query the server when the filter selection changes, but each individual query takes less time to run.

Test Plan

Locally. Will add tests.

Deployment Plan

Ideally ASAP

netlify[bot] commented 1 month ago

Deploy Preview for thriving-cassata-78ae72 canceled.

Name Link
Latest commit fa8285365df8dab8d382f02135be2a3b31414529
Latest deploy log https://app.netlify.com/sites/thriving-cassata-78ae72/deploys/671c657436b54300085dfaf6