search

sanity-io / sanity-algolia

Utilities for indexing Sanity documents in Algolia
MIT License
67 stars 16 forks source link

Unreliable updates due to synchronization issue between webhook received and sanity client fetch #5

Closed maxwellpopescu closed 3 years ago

maxwellpopescu commented 3 years ago

I have been using the sanity-algolia client for a while but I kept facing weird issues like the published change not being reflected in the algolia index.

Today I started using the 'visibility' function and noticed some problematic behavior when toggling a document's "isHidden" field on and off and publishing.

A lot of the time, the document fetched by sanity-algolia's webhookSync function did not have the updated isHidden value

So it seems that when the serverless function receives a webhook notification, webhookSync immediately fetches the document from Sanity but this fetched data still needs to be updated by sanity. To confirm this, I added a simple timer to delay the document fetching by 5 seconds: 

  const delay = async (ms: number) =>
    new Promise((res) => setTimeout(res, ms))
  await delay(5000)
  console.log('waited 5s')
  const result = await sanityAlgolia
    .webhookSync(sanity, req.body)
    .then(() => res.status(200).send('ok'))

This solved the issue, and so my algolia index is now being updated properly.

My useCdn is set to false, so I am unsure why Sanity is sending out stale data. Perhaps the webhook notification is sent before the changes are saved? A 5 second timer is a bit of a hack, so I am hoping there is a better way to deal with this issue.

runeb commented 3 years ago

Hi @maxwellpopescu ! Thanks for your submitted issue. The issue here seems to be that the hook is responding faster than Sanitys query indexer can catch up with committed patches. For listeners (https://www.sanity.io/docs/listening) Sanity support setting a flag to only be notified once a change is also visible in the query layer. For webhooks this flag is not yet available, so there are two possible solutions here.

  1. Wait some amount of time, like you have done
  2. Use the API that returns documents by id, istead of running a groq query. This will make some other things harder, like resolving references. That is a feature I'm most likely adding to this plugin soon.

I'll add your solution into this project so users don't have to do that manually in their serverless function.