search

watermarkchurch / wcc-contentful

An alternative to Contentful's contentful.rb ruby client, contentful_model, and contentful_rails gems all in one.
MIT License
2 stars 1 forks source link

Sync Engine: Exponential Backoff for Retries #265

Closed gburgett closed 2 years ago

gburgett commented 2 years ago

The Contentful CDN aggressively caches content, and it often happens that a Webhook will be sent and processed before the Sync API is able to provide the new content. This results in a Sync that does not include the entry indicated in the webhook (passed to the job via the up_to_id parameter.) In this case, the following lines appear in the logs, and the job is re-enqueued for 10 minutes later (Changed to 10 seconds in 2bc755a, not yet released)

INFO -- : [ActiveJob] [WCC::Contentful::SyncEngine::Job] [8447dd93-0be7-49e9-a7fb-66804f4a3fa4] WCC::Contentful::Event::SyncComplete
INFO -- : [ActiveJob] [WCC::Contentful::SyncEngine::Job] [8447dd93-0be7-49e9-a7fb-66804f4a3fa4] Synced 0 entries.  Next sync token:
  FEnCh...
INFO -- : [ActiveJob] [WCC::Contentful::SyncEngine::Job] [8447dd93-0be7-49e9-a7fb-66804f4a3fa4] Should enqueue again? [true]

10 minutes is far too long a time to wait. This PR more agressively hits the Sync API to try to catch the CDN update as soon as possible. Now, if the changed entry is not present on the Sync API yet, the SyncEngine::Job will retry after 1 second, then 2 seconds, then 4 seconds, for a total of 4 attempts over 7 seconds before giving up.

watermarkchurch-bot commented 2 years ago
1 Warning
:warning: No issue referenced - please create an issue describing a single unit of work and reference it using "closes #[the issue number]"

Generated by :no_entry_sign: Danger