New Data Load : Tisiphone abeona (spp: abeona & albifascia) collected specimen occurrence records (2002 & 2022)

[cha801p]

Ticket Update: April 4, 2024 (12 PM)

Issue: New Data Load (https://collections-test.ala.org.au/public/show/dr22445)

Solution: Successfully load the new dataset into biocache/prod

Actions Taken:

• [x] Data review
• [x] Answered data providers queries regarding images
• [x] Waiting for updated CSV

Ticket: https://support.ehelp.edu.au/a/tickets/198597

[cha801p]

Ticket Update: April 8, 2024 (6 PM)

Issue: New Data Load

Solution: Successfully load the new dataset into biocache/databox

Actions Taken: Successfully loaded the data on test

• [x] Data review
• [x] Columns deleted - licence
• [x] EventDate column reformatted
• [x] Empty columns deleted
• [x] DwcA created locally
• [x] Loaded the data on collectory-test
• [x] Ran preingestion on databox

Loaded data for review: Metadata: data

Logs: INFO [2024-04-08 07:44:50,962+0000] [main] au.org.ala.pipelines.beam.ALAUUIDMintingPipeline: As this is an initial load for this dataset, UUID change checks are not ran. INFO [2024-04-08 07:44:50,962+0000] [main] au.org.ala.pipelines.beam.ALAUUIDMintingPipeline: Pipeline complete. INFO [2024-04-08 07:44:50,962+0000] [main] au.org.ala.pipelines.beam.ALAUUIDMintingPipeline: Checking for backups to prune..../data/pipelines-data/dr22445/1/identifiers INFO [2024-04-08 07:44:50,963+0000] [main] au.org.ala.pipelines.beam.ALAUUIDMintingPipeline: Writing metrics..... INFO [2024-04-08 07:44:50,966+0000] [main] org.gbif.pipelines.common.beam.metrics.MetricsHandler: Trying to write pipeline's metadata to a file - /data/pipelines-data/dr22445/1/uuid-metrics.yml INFO [2024-04-08 07:44:50,968+0000] [main] org.gbif.pipelines.common.beam.metrics.MetricsHandler: Added pipeline metadata - newUuidsAttempted: 184, INFO [2024-04-08 07:44:50,994+0000] [main] org.gbif.pipelines.common.beam.metrics.MetricsHandler: Metadata was written to a file - /data/pipelines-data/dr22445/1/uuid-metrics.yml INFO [2024-04-08 07:44:50,995+0000] [main] au.org.ala.pipelines.beam.ALAUUIDMintingPipeline: Writing metrics written.

Status: Waiting for images to be received from the data provider Dataset is ready for review

[peggynewman]

I LOVE these tickets and thorough documentation! Thanks!

• [x] License needs to be added
• [x] Coordinate precision looks fishy, should be something like: 0.00001 (see Sharing data with the ALA article on location)
• [x] Occurrence status inferred from individual count flag set ... set occurrenceStatus to Present
• [x] Collection code/institution codes don't map to a provider map:

Supplied institution code "MQU, Sydney" Supplied collection code "MCV" (check with @rosemaryjoconnor about provider maps!) Is MCV an actual collection?

Thanks!

[cha801p]

Ticket Update: June 18, 2024 (6:30 PM)

Issue: Load data on Prod

Solution: Successfully ingested the new dataset onto prod

Actions Taken: Successfully loaded the data on prod

Data reviewed Columns deleted - licence Empty columns deleted DwcA created locally Loaded the data on collectory Ran preingestion on prod

Loaded data for review: Metadata: data

Logs: INFO [2024-06-18 06:48:00,056+0000] [main] au.org.ala.pipelines.beam.ALAUUIDMintingPipeline: Writing metrics..... INFO [2024-06-18 06:48:00,057+0000] [main] org.gbif.pipelines.common.beam.metrics.MetricsHandler: Trying to write pipeline's metadata to a file - /data/pipelines-data/dr22687/1/uuid-metrics.yml INFO [2024-06-18 06:48:00,058+0000] [main] org.gbif.pipelines.common.beam.metrics.MetricsHandler: Added pipeline metadata - newUuidsAttempted: 184, INFO [2024-06-18 06:48:00,078+0000] [main] org.gbif.pipelines.common.beam.metrics.MetricsHandler: Metadata was written to a file - /data/pipelines-data/dr22687/1/uuid-metrics.yml

Status: Waiting for indexing

[cha801p]

Data loaded on prod and email sent to the data provider. https://support.ehelp.edu.au/a/tickets/198597