Monitor Data Loads : 17/10/2022

[rosemaryjoconnor]

• [ ] After creation, add this issue to the Data Management project board.

Jenkins Access

• SSH:   ssh -L 9194:127.0.0.1:9194 aws-spark-quoll-master.ala
• Data resource loads: Data resource loads
• GBIF Loads: GBIF Loads

Reference

• Jenkins - Automated Data Load Schedule: Jenkins Schedule

---

Monitoring Schedule - automated jobs are to be monitored twice per week on Tuesday and Friday.

Please enter Week commencing date: 17/10/2022 Sandbox Monitoring

• Go to: Sandbox
• Check for uploaded datasets that look to be from 'real' users. Ignore Test or datasets that are for the spatial portal.
• Flag any datasets that need to be looked at.

Tuesday Monitoring

Data-resource-loads folder:

• [x] dp3903-biocollect-monthly
• [x] dr1411-inaturalist-weekly
• [ ] dr15863-perth-avh-weekly
• [ ] dr1902-questagame-weekly
• [x] dr368-bionet-weekly-delta
• [x] dr376-mel-avh-weekly

Friday Monitoring

GBIF folder:

• [x] Dwca-export-all
• [x] Dwca-package-for-gbif-all

Data-resource-loads folder:

• [ ] dr10574-ho-avh-weekly
• [x] dr15860-canb-avh-weekly
• [x] dr15867-cns-avh-weekly
• [x] dr15868-jct-avh-weekly
• [ ] dr2153-nzvh-avh-weekly
• [x] dr341-canb-anwc-weekly
• [x] dr7973-cPlatypus-weekly

There are also quarterly and monthly jobs that will need to be checked on the following days of the month:

3rd of month quarterly - Feb, May, Sept, Nov

• [ ] dr365,dr366-BDBSA-quarterly

5th day of month

• [ ] dr344-qm-full-load-monthly

28th day of month

• [ ] dp5183-obis-seamap-monthly