Problems creating replica indices

[Bruce773]

I ran into a number of problems while attempting to create replica indices with this plugin. In the end, I decided to remove the plugin and write my own script.

I was passing queries generated by mapping over a list of settings like this (the algoliaQueries was eventually passed to the queries field of this plugin):

// Primary index
algoliaQueries.push({
  forwardToReplicas: true,
  settings: {
    replicas: [
      `${indexName}_${lang.code}_most_popular`,
      `${indexName}_${lang.code}_newest`,
      `${indexName}_${lang.code}_oldest`,
      `${indexName}_${lang.code}_alpha`
    ],
    unretrievableAttributes: [
      ...commonUnretrievableAttributes,
      ...refinements.map(r => withPrefix(r.facetingAttributeName))
    ],
    attributesForFaceting: refinements.map(r =>
      withPrefix(r.facetingAttributeName)
    ),
    sortFacetValuesBy: "alpha"
  },
  transformer: ({ data }) => {
    const items = getContentItemsFromQueryResult(data);
    const result = items.map(addAlgoliaFields);
    return result;
  },
  query: query(lang.name),
  indexName: `${indexName}_${lang.code}`
});

I then had a separate script that created my replicas as detailed in this issue: https://github.com/algolia/gatsby-plugin-algolia/issues/41

The replica generation script ran first followed by the plugin's code that created the primary indices. However, halfway through the process (half of my indices still had _tmp tacked onto the end of them in the Algolia web portal), I would get an error saying AlgoliaSearch: cannot move with a primary/replica index as source. This error seemed to persist no matter what I did.

Eventually, I ended up just writing my own implementation of the plugin.

[Haroenv]

I'd like to see what changes you've made to the plugin, to be honest, it wasn't made with replicas in mind initially, maybe just a few waitTask on the right spot should do the job just fine?

[Bruce773]

My implementation was actually pretty simple... I just created a script that would build the replica indices then construct the primaries all based on the queries I was originally passing into gatsby-plugin-algolia:

const algoliasearch = require('algoliasearch');
const { algoliaQueries, algoliaReplicaIndices } = require('./algoliaQueries');

module.exports.syncAlgoliaSettings = async function() {
    const client = algoliasearch(id, key);

    for (const item of algoliaReplicaIndices) {
      const { indexName, settings } = item;
      client
        .initIndex(indexName)
        .setSettings(settings, {}, (_err, content) =>
          console.log('Algolia replica index: ', content),
        );
    }

    for (const item of algoliaQueries) {
      const { indexName, settings, query, transformer } = item;
      client
        .initIndex(indexName)
        .setSettings(settings, { forwardToReplicas: true, transformer, query }, (_err, content) =>
          console.log(_err, content),
        );
    }
};

[Bruce773]

@Haroenv I spent some time digging into the plugin code and I think I've got a solution. :) https://github.com/yogainternational/gatsby-plugin-algolia/commit/fc7a124c14d0779823ee64901c33b0571cbfa5c3

[Bruce773]

I kept getting an error saying A replica index is already replica of another index... I believe I tracked it down to the fact that replica indices were being added to _tmp indices... Once I filtered that out, it seemed to work. But, I'm still testing some things out.

[Haroenv]

Thanks for looking into this @Bruce773