WeedScan updates

[peggynewman]

Weedscan are providing a new field, ai_confidence. The fetcher should filter those >= 50%.

Collectory-test: https://collections-test.ala.org.au/public/show/dr21567 Collectory-Prod: https://collections.ala.org.au/public/show/dr21567

• [x] Run the WeedScan ingestion in the test environment. Provide a summary of the values in ai_confidence (10% bins, record count)

• [ ] Update the fetcher to filter those > 50%

• [ ] Run in test again and let Peggy know

• [ ] Wait for verification with the data provider and Peggy to get the API key for prod

• [ ] Update config with prod API key & Deploy fetcher

• [ ] Run in prod and let Peggy know

[cha801p]

Total records - 1890 below 50% - 908

[cha801p]

Ticket Update: 14 December, 2023 (3:15 PM)

Issue Identified: Update WeedScan fetcher and run on prod

Actions Taken:

• [x] Data has been refreshed and loaded on test

Waiting for Updates:

• [x] Images links are not working
• [x] API link used to download data is a test link (https://test.weedscan.org.au/api/1.0/Observation/DarwinCoreOutput/)
• [ ] Waiting for clarification on ai_confidence column filtration

Status: Waiting for the customer

[peggynewman]

• Image links should now work
• I have a new api key for their prod system: https://weedscan.org.au/api/1.0/Observation/DarwinCoreOutput/

[rosemaryjoconnor]

01/2024 GitHub:

• [x] github branch: https://github.com/AtlasOfLivingAustralia/preingestion/tree/feature/dr21567-Weedscan-%23997
• [x] Weedscan fetcher updated - updated logging info
• [x] baseconfig.py - new API key for Prod

[peggynewman]

Did we actually make a decision about filtering on the Confidence field before the break?

[rosemaryjoconnor]

No we didn't. Raj was also going to take a look at what had been done to date as he was on the email re this also.

[peggynewman]

Still working out what to set the confidence filter to.

[peggynewman]

Issue is stuck. Going back to the backlog.