Solr documents where the created field value is missing will now be treated as out of sync with portal catalog. This fixes an issue we had in production where documents that were only inserted partially in solr would not be picked up by diff as their modified field was provided correctly.
We assume the partial documents were created as following:
during an upgrade or other reindex operation we have queued an atomic update for the plone content which only updated a subset of the fields, e.g. by sending the following update command to solr
this solr document will have the same UID and the same modified date as plones portal_catalog, but is missing a lot of essential attributes. it would not be picked up in a solr diff however as we only compare modified and UID for performance reasons
The field created can be used as a good heuristic as it is most likely only added via the initial add when an object in plone is created, or only present when all object indices are reindexed without restricting the list of indices. Furthermore the created field should always be present and always be filled with a valid date. It is not changed during the lifetime of plone content under normal circumstances, thus we expect it is never part of an atomic update.
So if created is missing from a document something has gone wrong and that document needs reindexing.
Some basic smoke-tests for this change are provided in https://github.com/4teamwork/opengever.core/pull/6577. This seemed to be the fastest path to some tests as in ftw.solr we don't test against a real solr and the maintenance view seems fairly untested 🤔.
Solr documents where the
created
field value is missing will now be treated as out of sync with portal catalog. This fixes an issue we had in production where documents that were only inserted partially in solr would not be picked up bydiff
as theirmodified
field was provided correctly.We assume the partial documents were created as following:
UID
and the samemodified
date as plones portal_catalog, but is missing a lot of essential attributes. it would not be picked up in a solrdiff
however as we only comparemodified
andUID
for performance reasonsThe field
created
can be used as a good heuristic as it is most likely only added via the initial add when an object in plone is created, or only present when all object indices are reindexed without restricting the list of indices. Furthermore the created field should always be present and always be filled with a valid date. It is not changed during the lifetime of plone content under normal circumstances, thus we expect it is never part of an atomic update.So if
created
is missing from a document something has gone wrong and that document needs reindexing.Some basic smoke-tests for this change are provided in https://github.com/4teamwork/opengever.core/pull/6577. This seemed to be the fastest path to some tests as in
ftw.solr
we don't test against a real solr and the maintenance view seems fairly untested 🤔.For https://4teamwork.atlassian.net/browse/GEVER-795.