Display of data quality profile category broken

AtlasOfLivingAustralia / la-pipelines

Living Atlas Pipelines extensions

3 stars 4 forks source link

Display of data quality profile category broken #445

Closed brucehyslop closed 3 years ago

brucehyslop commented 3 years ago

The display of the profile category does not function if the filter contains multiple query terms.

Screen Shot 2021-06-24 at 5.46.39 pm.png

adam-collins commented 3 years ago

The display has additional issues:

view excluded records link does not do what it says.
Further information links display old links that work but have the old page titles.
- spatiallyValid links to https://github.com/AtlasOfLivingAustralia/ala-dataquality/wiki/geospatial_kosher
- -assertions:"PRESUMED_SWAPPED_COORDINATE" links to https://github.com/AtlasOfLivingAustralia/ala-dataquality/wiki/INVERTED_COORDINATES

javier-molina commented 3 years ago

Thanks @adam-collins , the items below are still work in progress and are being addressed as we speak.

Further information links display old links that work but have the old page titles.

spatiallyValid links to https://github.com/AtlasOfLivingAustralia/ala-dataquality/wiki/geospatial_kosher

-assertions:"PRESUMED_SWAPPED_COORDINATE" links to https://github.com/AtlasOfLivingAustralia/ala-dataquality/wiki/INVERTED_COORDINATES

javier-molina commented 3 years ago

Related issues #419 #456 have to be released together.

javier-molina commented 3 years ago

@adam-collins any further comment re: code changes or review in general, see comment above for how this needs to be tested.

alexhuang091 commented 3 years ago

issue filter value is broken @brucehyslop

alexhuang091 commented 3 years ago

And I remember there are explanations/links for userAssertions 50001 and 50005. Can they be put here?

nickdos commented 3 years ago

@javier-molina should this move back to "in progress" to fix the missing values highlighted in the comment above?

brucehyslop commented 3 years ago

Issues relating to the display of DQ filters and links are covered in #419

brucehyslop commented 3 years ago

The reason the links for userAssertions are not displaying on test is because there are no records with userAssertions all Solr shards. The link data relies on a request to biocache-service to retrieve the index field data.

https://biocache-ws-test.ala.org.au/ws/index/fields?fl=userAssertions

This endpoint triggers a Solr LUKE request which queries a single shard. If that shard that is chosen does have records that contain a field the field on not returned.

The links will be displayed sporadically depending on which shard is chosen.

Screen Shot 2021-08-19 at 3 26 12 pm

brucehyslop commented 3 years ago

The issue does not occur in production because records that contain userAssertions are distributed across all shards.

brucehyslop commented 3 years ago

The this is the same reason there is no link displayed for outlierLayerCount however this is because sampling on not being run on test.

The link does display on prod.