Currently additions/updates/deletions are handled by comparing the contents of mongodb vs. ADS classic, and any updates made to mongodb are then propagated to SOLR. Occasionally these updates do not make it successfully and therefore SOLR becomes out of sync wrt mongodb.
For instance as of today:
A) Records in ADS classic: 12,020,853
B) Records in mongodb: 12,020,776
C) Records in SOLR: 12,020,782
Mongodb (B) is synced with ADS classic every day, so the fact that A > B simply means that there are ingestion problems that should be fixed.
The fact that C > B means that SOLR contains some records that are now stale and should have been removed from its index but somehow that process failed.
The way to implement this may be as simple as adjusting the recipe that we use to detect SOLR missing records to handle deletions.
Currently additions/updates/deletions are handled by comparing the contents of mongodb vs. ADS classic, and any updates made to mongodb are then propagated to SOLR. Occasionally these updates do not make it successfully and therefore SOLR becomes out of sync wrt mongodb.
For instance as of today:
Mongodb (B) is synced with ADS classic every day, so the fact that A > B simply means that there are ingestion problems that should be fixed.
The fact that C > B means that SOLR contains some records that are now stale and should have been removed from its index but somehow that process failed.
The way to implement this may be as simple as adjusting the recipe that we use to detect SOLR missing records to handle deletions.