How to create/sync replicas?

[brettinternet]

There currently doesn't appear to be an API here to create replicas following an index creation/sync. Our transform is somewhat complicated and does some asynchronous actions outside of the Gatsby DataLayer, so we'd like to avoid repeating the queries/transforms with each replica. Here are two possible approaches to consider.

Assume we have an index and two replicas:

const indexName = ALGOLIA_INDEX_PREFIX + ALGOLIA_INDEX_NAME
const replicas = {
  DATE_DESC: `${indexName}_date_desc`,
  DATE_ASC: `${indexName}_date_asc`,
}

Option 1

We allow query items to not contain the "query" attribute, and assume it's a replica if the indexName is contained in some query that does have a "query" attribute. Notice in this example the replicas only contain an index name and settings.

// gatsby-config.js
const commonSettings = { // since forwardToReplicas isn't available
  searchableAttributes: ["description", "title", "content"],
  attributesToSnippet: ["content"],
  attributesForFaceting: ["category", "_tags"],
  distinct: true,
  attributeForDistinct: "url",
  ranking: [
    "typo",
    "geo",
    "words",
    "filters",
    "proximity",
    "attribute",
    "exact",
    "desc(date_timestamp)"
  ]
};

module.exports = {
  plugins: [
    {
      resolve: `gatsby-plugin-algolia`,
      options: {
        appId: env.ALGOLIA_APP_ID,
        apiKey: env.ALGOLIA_API_KEY,
        queries: [
          {
            query: postQuery,
            transformer: [Function],
            indexName,
            settings: {
              replicas: Object.values(replicas),
              ...commonSettings
            }
          },
          {
            indexName: replicas.DATE_DESC,
            settings: {
              ...commonSettings,
              ranking: [
                "desc(date_timestamp)",
                "typo",
                "geo",
                "words",
                "filters",
                "proximity",
                "attribute",
                "exact"
              ]
            }
          },
          {
            indexName: replicas.DATE_ASC,
            settings: {
              ...commonSettings,
              ranking: [
                "asc(date_timestamp)",
                "typo",
                "geo",
                "words",
                "filters",
                "proximity",
                "attribute",
                "exact"
              ]
            }
          }
        ]
      }
    }
  ]
};

Option 2

Or we add a completion hook to the library where the user may sync the replicas with algoliasearch JS library.

// gatsby-config.js
module.exports = {
  plugins: [
    {
      resolve: `gatsby-plugin-algolia`,
      options: {
        appId: env.ALGOLIA_APP_ID,
        apiKey: env.ALGOLIA_API_KEY,
        queries: [...],
        onComplete: syncSettings
      }
    }
  ]
};

What do you think about these approaches? What's your take on incorporating replica creation/sync in this library?

[brettinternet]

The completion hook is basically pointless since the user could just implement a their own onPostBuild in gatsby-node. From my tests Gatsby will run Algolia's plugin before the user's own onPostBuild.

This is apparently a viable workaround so I'll close this issue. Here's an example for future viewers, or we might document this in the README if this seems like a popular use case @Haroenv.

// gatsby-node.js
exports.onPostBuild = async function() {
  await syncAlgoliaSettings()
}

// algoliaSettings.js
exports.syncAlgoliaSettings- async function() {
  const client = algoliasearch(ALGOLIA_APP_ID, ALGOLIA_API_KEY);
  const indexSettingsTask = [
    {
      indexName,
      forwardToReplicas: true,
      settings: {
        replicas: Object.values(replicas),
        searchableAttributes: ["description", "title", "content"],
        attributesToSnippet: ["content"],
        attributesForFaceting: ["category", "_tags"],
        distinct: true,
        attributeForDistinct: "url",
        ranking: [
          "typo",
          "geo",
          "words",
          "filters",
          "proximity",
          "attribute",
          "exact",
          "desc(date_timestamp)"
        ]
      }
    },
    {
      indexName: replicas.DATE_DESC,
      settings: {
        ranking: [
          "desc(date_timestamp)",
          "typo",
          "geo",
          "words",
          "filters",
          "proximity",
          "attribute",
          "exact"
        ]
      }
    },
    {
      indexName: replicas.DATE_ASC,
      settings: {
        ranking: [
          "asc(date_timestamp)",
          "typo",
          "geo",
          "words",
          "filters",
          "proximity",
          "attribute",
          "exact"
        ]
      }
    }
  ];

  for (const task of indexSettingsTask) {
    const { indexName, forwardToReplicas = false, settings } = task;
    await new Promise((resolve, reject) =>
      client
        .initIndex(indexName)
        .setSettings(settings, { forwardToReplicas }, (err, content) => {
          if (err) reject(err);
          console.log(
            `✔ Successfully synced settings with Algolia index ${indexName}`
          );
          resolve(content);
        })
    );
  }
}

[Haroenv]

I wonder if we can do it instead here

https://github.com/algolia/gatsby-plugin-algolia/blob/748ea45f9a8cdd19d141746a35dbc189b42b1c62/gatsby-node.js#L59

and put forwardToReplicas as true if user asked for that in the settings.

By the way, setSettings is already a promise, so you don't need to promisify it :)