During a recent full reindex, it was discovered that indexing all LoC concepts via the ingestor service takes significantly longer than the maximum Lambda allowed execution time (see here for more info).
As a result, it does not make sense to keep a deployed version of the ingestor service in its current form. Rather than refactoring the service to get around this issue, we decided to remove the Lambda function and only run the service locally on the rare occasion when we need to do a full reindex.
This removes all ingestor service infrastructure from AWS and expands the documentation to explain the situation.
The Terraform changes in this PR have not been applied yet.
Have we considered potential risks?
There should be no associated risks, the ingestor service was not scheduled to run regularly and removing the infrastructure will have no impact on user experience.
What does this change?
https://github.com/wellcomecollection/platform/issues/5815
During a recent full reindex, it was discovered that indexing all LoC concepts via the
ingestorservice takes significantly longer than the maximum Lambda allowed execution time (see here for more info).As a result, it does not make sense to keep a deployed version of the
ingestorservice in its current form. Rather than refactoring the service to get around this issue, we decided to remove the Lambda function and only run the service locally on the rare occasion when we need to do a full reindex.This removes all
ingestorservice infrastructure from AWS and expands the documentation to explain the situation.The Terraform changes in this PR have not been applied yet.
Have we considered potential risks?
There should be no associated risks, the
ingestorservice was not scheduled to run regularly and removing the infrastructure will have no impact on user experience.