Cannot push markdown content to Algolia for Indexing

[bhupendra1011]

I was trying to integrate Alogolia Search with a Gatsby site using gatsby-plugin-algolia. Below is the code from gatsby-config to query markdown files content and push it to algolia for indexing .

require("dotenv").config({
  path: ".env",
})
const blogQuery = `query {
  allMarkdownRemark {
    nodes {
      frontmatter {
        title
      }
      excerpt
    }
  }
}
`
const queries = [
  {
    query: blogQuery,
    transformer: ({ data }) => data.allMarkdownRemark.nodes,
  },
]

module.exports = {

  plugins: [
    {
      resolve: "gatsby-plugin-algolia",
      options: {
        appId: process.env.ALGOLIA_APP_ID,
        apiKey: process.env.ALGOLIA_API_KEY,
        indexName: process.env.ALGOLIA_INDEX_NAME, // for all queries
        queries,
        cunkSize: 1000,
      },     
    ],
}

I have verified API keys in .env file also checked in graphQL playground that blogQuery returns results. Here is the result what I get in localhost:8000/__graphQL

When I run gatsby build I get below error :

I also tried to debug error in generated log, but could not figure out more :

Kindly let me know what I am missing as Algoia API keys are correct and graphQL query fetches records.

[Haroenv]

Not sure why this didn't throw a clear error (you're maybe not on the latest version), but id or objectID is needed in an object to be able to retrieve it correctly

[bhupendra1011]

Hey @Haroenv thanks for replying , but sorry I did not get which id /object id to be added. Below are my dependencies :

[Haroenv]

In the query you have no identifiers:

const blogQuery = `query {
  allMarkdownRemark {
    nodes {
      frontmatter {
        title
      }
      excerpt
    }
  }
}
`

it should be e.g.

const blogQuery = `query {
  allMarkdownRemark {
    nodes {
      objectID: id
      frontmatter {
        title
      }
      excerpt
    }
  }
}
`

not sure if that node has an id field, but I think you get the idea

[bhupendra1011]

Thanks @Haroenv by mentioning objectID in query solved the issue. Didn't mention it as the docs say ObjectID if this field is absent, it's going to

be inserted by Algolia automatically

    # and will be less simple to update etc.

can we add a user friendly message if objectID is missing for indexing data

[Haroenv]

There is an error when the objectID is missing, so it might be triggered wrongly. Do you have a reproduction I can try out where this error isn't thrown? https://github.com/algolia/gatsby-plugin-algolia/blob/master/gatsby-node.js#L109

[daniellangnet]

Not sure it's helpful @Haroenv but I also just ran into this issue and an objectID is definitely required or otherwise the indexing will silently fail without throwing any errors.

[Haroenv]

hmm, that's really weird, can you reproduce with a PR to the example @daniellangnet? I'd like to fix this, but haven't seen the silent-error myself. Thanks!

[daniellangnet]

@Haroenv for testing, I just wrote a simple GraphQL query but didn't actually use the result and instead added a transformer function that returns this:

return [
    {
      title: "First Entry",
      slug: "/first/"
    },
    {
      title: "Second Entry",
      slug: "/second/"
    }
  ];

When running this, the index in Algolia stays empty and I can see in the console log that the postBuild triggers. No error messages.

Then, if I change the transformer to return this:

  return [
    {
      objectID: "1",
      title: "First Entry",
      slug: "/first/"
    },
    {
      objectID: "2",
      title: "Second Entry",
      slug: "/second/"
    }
  ];

it then works fine and the two records get added to the index.

[Haroenv]

When I do exactly that, I get

success Building static HTML for pages — 2.000 s
⠁ Algolia: 1 queries to index
⠄ index to AlgoliaAlgolia: query #1: executing query
error failed to index to Algolia. Query results do not have 'objectID' or 'id' key
error Command failed with exit code 1.

I wonder what you do differently, could you still make a GitHub repo please?

[anyav77]

I have a similar issue. I've been following the tutorial

The console returns the message: [HMR] connected I don't see any errors.

Here is my config file:

require('dotenv').config({
  path: `.env.production`,
});

const blogQuery = `
{
  allMarkdownRemark {
    nodes {
      objectID: id
      frontmatter {
        description
        title
      }
      excerpt
      html
    }
  }
}
`

const queries = [
  {
    query: blogQuery,
    transformer: ({ data }) => data.allMarkdownRemark.nodes, // optional
    indexName: process.env.ALGOLIA_INDEX_NAME, // overrides main index name, optional - used for multiple indexes
    //settings: {
      // optional, any index settings
      // Note: by supplying settings, you will overwrite all existing settings on the index
    //},
    // matchFields: ['slug', 'modified'], // Array<String> overrides main match fields, optional
  },
];

module.exports = {

  siteMetadata: {
    title: `Gatsby Starter Blog`,
    author: {
      name: `Kyle Mathews`,
      summary: `who lives and works in San Francisco building useful things.`,
    },
    description: `A starter blog demonstrating what Gatsby can do.`,
    siteUrl: `https://gatsbystarterblogsource.gatsbyjs.io/`,
    social: {
      twitter: `kylemathews`,
    },
  },
  plugins: [
    `gatsby-plugin-image`,
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        path: `${__dirname}/content/blog`,
        name: `blog`,
      },
    },
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        name: `images`,
        path: `${__dirname}/src/images`,
      },
    },
    {
      resolve: `gatsby-transformer-remark`,
      options: {
        plugins: [
          {
            resolve: `gatsby-remark-images`,
            options: {
              maxWidth: 630,
            },
          },
          {
            resolve: `gatsby-remark-responsive-iframe`,
            options: {
              wrapperStyle: `margin-bottom: 1.0725rem`,
            },
          },
          `gatsby-remark-prismjs`,
          `gatsby-remark-copy-linked-files`,
          `gatsby-remark-smartypants`,
        ],
      },
    },
    `gatsby-transformer-sharp`,
    `gatsby-plugin-sharp`,
    // {
    //   resolve: `gatsby-plugin-google-analytics`,
    //   options: {
    //     trackingId: `ADD YOUR TRACKING ID HERE`,
    //   },
    // },
    {
      resolve: `gatsby-plugin-feed`,
      options: {
        query: `
          {
            site {
              siteMetadata {
                title
                description
                siteUrl
                site_url: siteUrl
              }
            }
          }
        `,
        feeds: [
          {
            serialize: ({ query: { site, allMarkdownRemark } }) => {
              return allMarkdownRemark.nodes.map(node => {
                return Object.assign({}, node.frontmatter, {
                  description: node.excerpt,
                  date: node.frontmatter.date,
                  url: site.siteMetadata.siteUrl + node.fields.slug,
                  guid: site.siteMetadata.siteUrl + node.fields.slug,
                  custom_elements: [{ "content:encoded": node.html }],
                })
              })
            },
            query: `
              {
                allMarkdownRemark(
                  sort: { order: DESC, fields: [frontmatter___date] },
                ) {
                  nodes {
                    excerpt
                    html
                    fields {
                      slug
                    }
                    frontmatter {
                      title
                      date
                    }
                  }
                }
              }
            `,
            output: "/rss.xml",
          },
        ],
      },
    },
    {
      resolve: `gatsby-plugin-manifest`,
      options: {
        name: `Gatsby Starter Blog`,
        short_name: `GatsbyJS`,
        start_url: `/`,
        background_color: `#ffffff`,
        theme_color: `#663399`,
        display: `minimal-ui`,
        icon: `src/images/gatsby-icon.png`, // This path is relative to the root of the site.
      },
    },
    `gatsby-plugin-react-helmet`,
    `gatsby-plugin-gatsby-cloud`,
    // this (optional) plugin enables Progressive Web App + Offline functionality
    // To learn more, visit: https://gatsby.dev/offline
    // `gatsby-plugin-offline`,
    {
      // This plugin must be placed last in your list of plugins to ensure that it can query all the GraphQL data
      resolve: `gatsby-plugin-algolia`,
      options: {
        appId: process.env.ALGOLIA_APP_ID,
        // Use Admin API key without GATSBY_ prefix, so that the key isn't exposed in the application
        // Tip: use Search API key with GATSBY_ prefix to access the service from within components
        apiKey: process.env.ALGOLIA_API_KEY,
        indexName: process.env.ALGOLIA_INDEX_NAME, // for all queries
        queries,
        chunkSize: 10000, // default: 1000
        // settings: {
        //   // optional, any index settings
        //   // Note: by supplying settings, you will overwrite all existing settings on the index
        // },
        // enablePartialUpdates: true, // default: false
        // matchFields: ['slug', 'modified'], // Array<String> default: ['modified']
        // concurrentQueries: false, // default: true
        // skipIndexing: true, // default: false, useful for e.g. preview deploys or local development
        // continueOnFailure: false // default: false, don't fail the build if algolia indexing fails
      },
    },

  ],
}

[Haroenv]

@anyav77 if the console says [HMR connected], that means you are running Gatsby dev, not Gatsby build. Indexing only happens at the build stage.

[anyav77]

Thank you! I was actually missing the dotenv plugin. Running npm install dotenv fixed the issue.

[Haroenv]

Glad to hear it's solved @anyav77! I'll close this issue now as it's not really focused on a single problem. Please open new issues if you have a problem!