search

AtlasOfLivingAustralia / la-pipelines

Living Atlas Pipelines extensions
3 stars 4 forks source link

invalid qa field being downloaded #491

Closed alexhuang091 closed 3 years ago

alexhuang091 commented 3 years ago

run this query

https://nci3-biocache-service.ala.org.au/ws/occurrences/offline/download?disableAllQualityFilters=true&fields=decimalLatitude,decimalLongitude,eventDate,scientificName,taxonConceptID,recordID,dataResourceName&qa=alex,nick&emailNotify=false&sourceId=2004&reasonTypeId=10&email=hua091@csiro.au&dwcHeaders=true&fq=(lsid:urn:lsid:biodiversity.org.au:afd.taxon:51dc6bc2-9165-4fc3-a374-1010ac3037d0)&fq=(year:"1930")

with qa=alex,nick

in SearchDAOImpl::addPostProcessingFields, alex is mapped to alex, nick is mapped to nick so there will be 1 column for alex and 1 column for nick

downloadHeaders.qaIds = Arrays.stream(downloadParams.getQa().split(","))
     .map((qa) -> fieldMappingUtil.translateFieldValue("assertions", qa))
     .toArray(String[]::new);

so fieldMappingUtil.translateFieldValue needs a change to filter out all invalid qa values?

djtfmartin commented 3 years ago

Arguably this isnt really an issue. If the user requests columns for fields we don't have indexed, then blank column values seems one acceptable outcome. The other outcome would be to throw an error. A case can be made for both, but I'm happy with the current functionality.