Jenkins DwCA - QM Data Load (Monthly)

AtlasOfLivingAustralia / data-management

Data management issue tracking

7 stars 0 forks source link

Jenkins DwCA - QM Data Load (Monthly) #679

Open sadeghim opened 3 years ago

sadeghim commented 3 years ago

Name	Schedule	Schedule comments	Date last run	DR uid	DR name	DR record count?	DwCA?	Broad description of job
QM Data Load (Monthly)	Monthly	H 22 28	28-January-2021	dr344	Queensland Museum provider for OZCAM	827,947	No	Monthly QM Data Load.

patkyn commented 3 years ago

Jenkins job set up and data ingested. However, dwca-dr job may need to incorporate duplicate key detection

peggynewman commented 3 years ago

Still needs to be scheduled in pipelines. Previous schedule 28th of each month at 10pm.

peggynewman commented 3 years ago

Ran this in pipelines/biocache-test on the 14th June. aws-jenkins load used to run on the 28th. we migrated on the 26th, so it's time to run this job again now.

patkyn commented 3 years ago

Opened an issue related to a multiline record that caused it to be split into 2 records when converting from csv to dwca. https://github.com/AtlasOfLivingAustralia/data-management/issues/690