Enable infinite scrolling with page queries

[devejlee]

Summary

I am a novice Gatsby user migrating my WordPress website to Gatsby with an edge case. I have separate pages for blog posts of different categories. These pages are generated with different page queries because Gatsby does not allow dynamic GraphQL, so I can't have one unified source of GraphQL to generate pages that use different frontmatter.

I have around 166 blog posts in total. About 50+ are part of the "interviews" category, so I dedicate a separate "interviews" page for these blog posts. But because there are 50+ posts, users have to scroll through all 50+ posts to get to the bottom of the page. I was hoping to be able to paginate them with an infinite scroll feature, but no gatsby tutorial nor answer from online forums deals with my edge case.

I understand gatsbygram solves something similar to my problem. But I cannot replicate its features for my project which is bigger in scale.

Basic example

An example of my "interviews" blog page from my blog. I have limited the number of posts to 12 so users wouldn't have to scroll 50+ posts to reach to the bottom of the page, but this causes older posts to be hidden from the user, which defeats the purpose of my blog.

import React from "react"
import { Link, graphql } from "gatsby"
import Layout from "../components/layout"
import PageBreadcrumb from "../components/pageBreadcrumbs"
import Img from "gatsby-image"

export default ({ data }) => {
  return (
    <Layout>
      <PageBreadcrumb crumbs={ [ 'Home', 'Interviews' ] } />
      <div className="categoryPostContainer">
        {data.allMarkdownRemark.edges.map(({ node }) => (
          <figure className="categoryPost" key={node.id}>
            <Link to={node.fields.slug}>
            <Img className="categoryPostImg" fluid={node.frontmatter.featuredImage.childImageSharp.fluid} />
            <figcaption>
              <h2>{node.frontmatter.category}</h2>
              <h3>
                {node.frontmatter.title}
              </h3>
              <h4>{node.frontmatter.date}{" "}by{" "}<span>{node.frontmatter.author}</span>{" "}</h4>
              <p>{node.excerpt}</p>
            </figcaption>
            </Link>
          </figure>
        ))}
      </div>
    </Layout>
  )
}

export const query = graphql`
  query {
    allMarkdownRemark(
      sort: { fields: [frontmatter___date], order: DESC }, limit: 12
      filter: {frontmatter: {category: {eq: "Interviews"}}}
    ) {
      totalCount
      edges {
        node {
          id
          frontmatter {
            title
            author
            date(formatString: "DD MMMM, YYYY")
            featuredImage {
              childImageSharp {
                fluid(maxWidth: 800) {
                  ...GatsbyImageSharpFluid
                }
              }
            }
          }
          fields {
            slug
          }
          excerpt
        }
      }
    }
  }
`

From my gatsby-node.js

const path = require(`path`)
const { createFilePath } = require(`gatsby-source-filesystem`)

exports.onCreateNode = ({ node, getNode, actions }) => {
  const { createNodeField } = actions
  if (node.internal.type === `MarkdownRemark`) {
    const slug = createFilePath({ node, getNode, basePath: `pages` })
    createNodeField({
      node,
      name: `slug`,
      value: slug,
    })
  }
}

exports.createPages = async ({ graphql, actions }) => {
  const { createPage } = actions
  const result = await graphql(`
    query {
      allMarkdownRemark {
        edges {
          node {
            fields {
              slug
            }
          }
        }
      }
    }
  `)

  result.data.allMarkdownRemark.edges.forEach(({ node }) => {
    createPage({
      path: node.fields.slug,
      component: path.resolve(`./src/templates/blog-post.js`),
      context: {
        // Data passed to context is available
        // in page queries as GraphQL variables.
        slug: node.fields.slug,
      },
    })
  })
}

Motivation

There are people like me who use separate page queries to make multiple lists of blog posts. Not all blogs have one list of blog posts. Gatsby pagination seems to be made for only one list of blog posts in mind, not for people like me who are migrating multiple lists of blog posts from an old CMS.

[herecydev]

You can use gatsby with some simple pagination to actually create multiple pages and load between them

However if you want an infinite scroll, then you're effectively lazy loading. You can have gatsby ssr the first bunch of blogs and then fetch calls to retrieve more. The reason you can't find this in gatsby is because that's a react issue at that point. You would also need a server to load from as well.

Hope that helps

[devejlee]

I see. It seems like pagination would be more easier to implement than infinite scrolling. I wasn't able to find any resource for pagination for multiple pages without a blog-list template online. Do you have any recommendations?

Doing more research on my own, it seems gatsby ssr will be more easy to implement than processing the gatsby-node.js soup of pagination.

[LekoArts]

I have separate pages for blog posts of different categories. These pages are generated with different page queries because Gatsby does not allow dynamic GraphQL, so I can't have one unified source of GraphQL to generate pages that use different frontmatter.

Can you explain that further? Gatsby is able to handle different frontmatters in the same GraphQL query (e.g. allMarkdownRemark).

However if you want an infinite scroll, then you're effectively lazy loading. You can have gatsby ssr the first bunch of blogs and then fetch calls to retrieve more.

This would be against Gatsby's paradigm as this would be fetched on the client and not pre-rendered. I would advise against it. If you absolutely want infinite scrolling, here's an example: https://gatsby-starter-infinite-scroll.baobab.fi/

I wasn't able to find any resource for pagination for multiple pages without a blog-list template online. Do you have any recommendations?

You can follow https://www.gatsbyjs.org/docs/adding-pagination/ and adapt it to create multiple lists. It's 100% doable.

---

On a personal note: I'd show all results and add a tag/category filter or a search. The site will most likely load quick enough without pagination.

[dandv]

There are people like me who use separate page queries to make multiple lists of blog posts.

Are you trying to do something similar to this?

[devejlee]

Can you explain that further? Gatsby is able to handle different frontmatters in the same GraphQL query (e.g. allMarkdownRemark).

I have 9 pages in my blog. Each page is rendered through page queries that search for different frontmatter. For the example above, the "Interviews" page filters for the "Interviews" category in allMarkdownRemark. For another "School" page, the page query filters for the "School" category and so on for all 9 pages.

Because of these 9 unique page queries, I cannot use the standard Gatsby pagination that depends on ONE source of GraphQL queries in a template file, because I have 9 sources in 9 separate page files.

I can't filter 9 unique page queries from ONE source of GraphQL because, as far as I know, Gatsby does not support dynamic GraphQL where I could filter for an "Interviews" category when I click on my "Interviews" page from my navigation.

To summarize, I need 9 different paginations for 9 different page queries whereas Gatsby (as far as I know) only offers a single source of pagination.

I will try to deploy my project via netlify at a later time for everyone to see if my explanation is still unclear.

[devejlee]

For future reference, I found a satisfying solution to my problem by simplifying some gatsbygram code.

I realized the genius of gatsbygram was to slice the number of posts that were rendered and customized the code to fit my needs down below

import React from "react"
import { Link, graphql } from "gatsby"
import Layout from "../components/layout"
import PageBreadcrumb from "../components/pageBreadcrumbs"
import Img from "gatsby-image"

export default class extends React.Component {
    constructor(props) {
    super(props)
        let postsToShow = 12        
    this.state = {
      showingMore: postsToShow > 12,
      postsToShow,
    }
    }

    render() {
        const posts = this.props.data.allMarkdownRemark.edges
        const index = this.state.postsToShow;
        console.log(this.props.data.allMarkdownRemark.edges.length)
        return (
            <Layout>
                <PageBreadcrumb crumbs={ [ 'Home', 'Test' ] } />
                <div className="categoryPostContainer">
                    {posts.slice(0, index).map(({ node }) => (
                        <figure className="categoryPost" key={node.id}>
                            <Link to={node.fields.slug}>
                            <Img className="categoryPostImg" fluid={node.frontmatter.featuredImage.childImageSharp.fluid} />
                            <figcaption>
                                <h2>{node.frontmatter.category}</h2>
                                <h3>
                                    {node.frontmatter.title}
                                </h3>
                                <h4>{node.frontmatter.date}{" "}by{" "}<span>{node.frontmatter.author}</span>{" "}</h4>
                                <p>{node.excerpt}</p>
                            </figcaption>
                            </Link>
                        </figure>
                    ))}
                    {this.state.postsToShow < this.props.data.allMarkdownRemark.edges.length &&
                    <div>
                        <button onClick={() => {
                            this.setState({
                                postsToShow: this.state.postsToShow + 12,
                            })
                        }}>Load More</button>
                    </div>
                }
                </div>
            </Layout>
        )
    }
}

export const query = graphql`
  query {
    allMarkdownRemark(
      sort: { fields: [frontmatter___date], order: DESC }
      filter: {frontmatter: {category: {eq: "Interviews"}}}
    ) {
      totalCount
      edges {
        node {
          id
          frontmatter {
            title
            author
            date(formatString: "DD MMMM, YYYY")
            featuredImage {
              childImageSharp {
                fluid(maxWidth: 800) {
                  ...GatsbyImageSharpFluid
                }
              }
            }
          }
          fields {
            slug
          }
          excerpt
        }
      }
    }
  }
`

[popoviciandrei]

this is bad, to load all the blogs in memory on page load. For a few blogs is fine, but if the number of blogs is growing.. then that initial page is will grow and grow... almost imposible to load on mobiles/ slow networks your site then.