[ENG-4633] trove_indexcard mitigations (small)

CenterForOpenScience / SHARE

SHARE is building a free, open, data set about research and scholarly activities across their life cycle.

http://share-research.readthedocs.io/en/latest/index.html

Apache License 2.0

100 stars 58 forks source link

[ENG-4633] trove_indexcard mitigations (small) #807

Closed aaxelb closed 1 year ago

aaxelb commented 1 year ago

skip the osfmap:contains property when indexing (e.g. when indexing metadata for a project, ignore metadata about the files it contains)
remove some unused fields from the elastic mapping (and the unused code that used them)
consolidate iri values into fewer nested_iri documents (in part thanks to those removed fields)

these will help avoid hitting the nested-doc limit of 10000 and (perhaps only slightly) speed up search queries

coveralls commented 1 year ago

coverage: 90.157% (-0.008%) from 90.165% when pulling cc2fc37fa404d32b12837258b9563750e56b6380 on aaxelb:fix/less-nest into a95bfd2f20b5b431a9a0355aedb47ea17954edc3 on CenterForOpenScience:develop.