search

AtlasOfLivingAustralia / biocache-store

Occurrence processing, indexing and batch processing
Other
7 stars 24 forks source link

Make hardcoded field lists configurable #309

Closed ansell closed 2 years ago

ansell commented 5 years ago

There are numerous hardcoded field lists that should be configurable to make biocache-store as flexible as possible. In one case, the exact order and size of the hardcoded field list in one method needs to match the order and size of another hardcoded field list in another method.

https://github.com/AtlasOfLivingAustralia/biocache-store/blob/master/src/main/scala/au/org/ala/biocache/index/SolrIndexDAO.scala#L1682-L1897

https://github.com/AtlasOfLivingAustralia/biocache-store/blob/master/src/main/scala/au/org/ala/biocache/index/IndexDAO.scala#L210-L392

https://github.com/AtlasOfLivingAustralia/biocache-store/blob/master/src/main/scala/au/org/ala/biocache/index/IndexDAO.scala#L676-L823

https://github.com/AtlasOfLivingAustralia/biocache-store/blob/master/src/main/scala/au/org/ala/biocache/index/IndexDAO.scala#L1065-L1375

https://github.com/AtlasOfLivingAustralia/biocache-store/blob/master/src/main/scala/au/org/ala/biocache/index/IndexDAO.scala#L1413-L1658

If there are specific operations required for fields, the operation should be encoded into the configuration file and interpreted inside of biocache store as necessary. For example, getRawOrDefault getRawOrParsed getParsedOrDefault getParsedOrRaw getCoordinate getTime could also be operations that are encoded in the configuration file and interpreted, rather than having to encode them as part of a hardcoded Scala program.

brucehyslop commented 2 years ago

biocache-store has been replaced by pipelines.