Data load : Museums Victoria update

[peggynewman]

Museums Victoria have FTPd through new data to the upload server. I think they have their own Jenkins job.

• [x] After creation, add this issue to the Data Management project board https://github.com/orgs/AtlasOfLivingAustralia/projects/9

• [x] Upload the dataset to collections.ala.org.au

• [x] Load/sample/process dataset http://aws-scjenkins.ala:9193/job/Parameterised%20Load%20Sample%20Process/

• [x] Note the statistics for total and new records from the end of the log file for the load by pasting the relevant lines here:

  Records loaded into the system: 80423, records skipped: 0, new records: 6065

• [x] Wait for next complete reindex

• [x] Check that the number of records accessible match the loaded numbers using https://biocache.ala.org.au/occurrence/search?q=data_resource_uid:drNNN

• [ ] Ask the data provider to review the new records using the same URL

[charvolant]

We are generating a lot of errors of the form Remote media store host not recognised: file:/data/biocache-load/dr342/489389.jpg for uuid=5331afad-22fb-4b51-80d7-55689c702fd3 Also, there are a number of lines of the form Only loading multimedia for specific core record types, starRecord.core().rowType()=dwc:Occurrence (dwc:Occurrence should be acceptable)

[charvolant]

The only loading message seems to be done for anything. Remote media store host not recognised simply seems to be logging that the image isn't coming from the image-service and therefore isn't part of an endless loop. Not sure why both of these are at info level. Manual checking of a few records seems to show that things are as they should be.

[charvolant]

Also, while I'm about it, why does every record I look at, no matter what the dataset, turn out to be on Lord Howe Island? I must be strangely attracted to the place.