search

AtlasOfLivingAustralia / ala-name-matching

Atlas name matching API and index generation
Other
10 stars 13 forks source link

Fix for wrong name matches when rank is filled #228

Open vjrj opened 2 months ago

vjrj commented 2 months ago

We discovered recently that some species are incorrectly matched when the rank is filled in the occurrence and correctly matched when it's not.

More info in: https://atlaslivingaustralia.slack.com/archives/CCTFGEU1G/p1719471737011039

Also described in: https://github.com/AtlasOfLivingAustralia/Taxonomic-Issues-Register_new/issues/161

More than the simple Eucalyptus test included in this PR, I did a new tool inside the la-toolkit to compare some random occurrences in a LA portal against the equivalent occurrences in GBIF.org.

Testing ~400 occurrences of our ~50M records with an ala-namematching-service without patching:

image

image

and with an ala-namematching-service using this patch:

2024-07-12_02-47

2024-07-12_02-43

And the last report of issues:

https://datos.gbif.es/others/202407120244_la_gbif_comparative_issues_report.html

As you can see that the number of differences and nulls in the scientificNames are significantly lower.

This comparative tool will help us also to debug aligning issues in pipelines.