Add an ability to perform complex async work

[kdichev]

my use case needs to fetch a few datasets prior to creating the index, which this plugin does not allow me to do.

I modified the plugin to accept an exported function that the product developer can fetch data and return the index to the plugin to upload to algolia.

https://github.com/kdichev/gatsby-plugin-algolia-index

// gatsby-config.js
module.exports = {
  plugins: [
    {
      resolve: `gatsby-plugin-algolia`,
      options: {
        appId: process.env.ALGOLIA_APP_ID,
        apiKey: process.env.ALGOLIA_API_KEY,
        chunkSize: 10000, // default: 1000
        path: 'path/to/custom/indexing-config-name' // optional, defaults to [algolia-index-config.js] in root
      },
    },
  ],
};

// algolia-index-config.js
module.exports = async (/* graphql */) => {
  // can do async work by using graphql client
  // can do index transforms
  return [{ indexName: 'index1', indexData: [{ title: 'index1' }] }, ...]
}

Let me know if you think this can be added to this plugin. @Haroenv

[Haroenv]

What's the reasoning you went for moving this into the external file, and not making people require it and pass it as reference? That way you're saving some complexity I think

[kdichev]

This is just what my gut feeling went with, no apparent reason. I think I was thinking that the way gatsby exposes its api's is quite neat

[Haroenv]

Which other places accept a string as a file? 🤔

[kdichev]

Actually when I think about it I do not remember where exactly did I get this idea from... 🔥 but I think you get the point of this proposal?

[kdichev]

I am actually requiring the config in the plugin itself but added a way to override the path to it 🤷‍♂️

[Haroenv]

I think it's pretty cool, but I don't currently have the bandwidth to figure out how to make both ways of customizing live together in "peace". I'd be interested if you have a PR to see what you should change

[brettinternet]

@Haroenv How about awaiting the transformer function for asynchronous workloads on the data? We use popularity data gathered from Google Analytics based on slugs provided by our CMS, which requires reducing the data asynchronous.

[Haroenv]

Since 0.4.0 you can do any async work inside the transformer. Thanks @brettinternet !