search

AtlasOfLivingAustralia / data-management

Data management issue tracking
7 stars 0 forks source link

TERN - New API and Fetcher #933

Open rosemaryjoconnor opened 1 year ago

rosemaryjoconnor commented 1 year ago

TERN API - about 45K plant preserved specimen records and put them into the ALA.

rosemaryjoconnor commented 1 year ago

New dr required as databox/test was refreshed from Prod

Fetcher has been written New species endpoint in API

rosemaryjoconnor commented 1 year ago

New data and slightly updated API available. Data now loaded in databox - dr19815 Have emailed TERN with an update. See below.

Fetcher code available in new branch in develop/dr19815-TERN

Status update There is some new data and a slightly updated API available. I've extracted the data and uploaded to our test system. I

The dataset can be viewed here: https://biocache-test.ala.org.au/occurrences/search?q=data_resource_uid%3Adr19815&qualityProfile=ALA

LocationID The API still does not include a URL to ecoplots for the LocationID. Asked re when it might be available. We can build the URL but as discussed previously it is better that it comes from the API to eliminate the chance of error if anything changes in the API.

Confirmation of filters Based on the search function in https://ecoplots.tern.org.au/search](https://ecoplots.tern.org.au/search?dataset=ausplots the filters currently applied are:

Data source name: TERN Surveillance Monitoring Feature Type: plant specimen Parameter: taxon

Parameter: taxon results in 48,364 records: https://ecoplots.tern.org.au/search?dataset=ausplots&feature_type=plant-specimen&observed_property=tern-cv%2F70646576-6dc7-4bc5-a9d8-c4c366850df0#/

Parameter: scientific name results in 48,364 records: https://ecoplots.tern.org.au/search?dataset=ausplots&feature_type=plant-specimen&observed_property=tern-cv%2F56195246-ec5d-4050-a1c6-af786fbec715

Asked for confirmation of which is preferred taxon or scientific name

peggynewman commented 9 months ago

Review of Additional Properties: Core ID: value is already mapped to Event ID . Remove. Feature ID: map to https://dwc.tdwg.org/terms/#dwc:otherCatalogNumbers ParentFeatureID is already mapped to EventID. Remove. ParentID is already mapped to parentEventID. Remove Parent site id/parent site name/project/result unit/ result unit id: are any of these values populated? remove all of these that are empty PointpositionwithinplotID is already mapped to georeferenceRemarks

cha801p commented 9 months ago

Ticket Update: 20 February, 2024 (11:30 AM)

Issue Identified: TERN dataset load on biocache

Actions Taken: TERN dataset loaded on test environment

Data links: biocache-test: https://biocache-test.ala.org.au/occurrences/search?fq=dataResourceUid:dr19843 events.test: https://events.test.ala.org.au/?datasetKey=dr19843

Status: Once confirmed by data provider, data will be loaded on prod

peggynewman commented 8 months ago

Feedback has come through from Data provider. I'll review it.