4teamwork / ftw.solr

Solr integration for Plone
5 stars 5 forks source link

Treat docs with no `created` field as out of sync in solr `diff` and `sync`. #173

Closed deiferni closed 3 years ago

deiferni commented 3 years ago

Solr documents where the created field value is missing will now be treated as out of sync with portal catalog. This fixes an issue we had in production where documents that were only inserted partially in solr would not be picked up by diff as their modified field was provided correctly.

We assume the partial documents were created as following:

The field created can be used as a good heuristic as it is most likely only added via the initial add when an object in plone is created, or only present when all object indices are reindexed without restricting the list of indices. Furthermore the created field should always be present and always be filled with a valid date. It is not changed during the lifetime of plone content under normal circumstances, thus we expect it is never part of an atomic update.

So if created is missing from a document something has gone wrong and that document needs reindexing.

Some basic smoke-tests for this change are provided in https://github.com/4teamwork/opengever.core/pull/6577. This seemed to be the fastest path to some tests as in ftw.solr we don't test against a real solr and the maintenance view seems fairly untested 🤔.

For https://4teamwork.atlassian.net/browse/GEVER-795.

deiferni commented 3 years ago

sry guys i can't select the gever team as reviewers, so i'm pulling you in directly