Currently the indexer service does not have any retry/error handling when calling the Notifier and SearchIndexer modules. Any error causes the celery task running the indexing service to error out.
We should assume that transient failures will happen at some point when attempting to send mobile notifications or indexing the data in the search engine.
In the short-term, one idea could be to implement retries with exponential back-off and a maximum number of retries before giving up. In the longer-term, a possibility is to have separate task queues for each type of indexing module (notifier, search , etc...) in order to provide failure isolation.
Currently the indexer service does not have any retry/error handling when calling the Notifier and SearchIndexer modules. Any error causes the celery task running the indexing service to error out.
We should assume that transient failures will happen at some point when attempting to send mobile notifications or indexing the data in the search engine.
In the short-term, one idea could be to implement retries with exponential back-off and a maximum number of retries before giving up. In the longer-term, a possibility is to have separate task queues for each type of indexing module (notifier, search , etc...) in order to provide failure isolation.