Soft delete Fossil data resources

[rosemaryjoconnor]

deregister from GBIF (untick Should be shared with GBIF ? : yes) find out what the 'Delete from GBIF' button does mark data resource as private remove AVRO

[peggynewman]

https://collections.ala.org.au/public/show/dr2290 Fossil Birds https://collections.ala.org.au/public/show/dr2289 Database of Australasian Vertebrate Occurrences https://collections.ala.org.au/public/show/dr2342 Fossil Fish

[peggynewman]

Clicked 'Delete from GBIF' Then unticked 'is shareable with GBIF' lost the GBIF registry key in collectory, but retained below Set it to private

dr2290 | https://www.gbif.org/dataset/c9d02825-4ff4-4483-bdd6-7bce652826d3 dr2289 | https://www.gbif.org/dataset/d3e6fa33-75a8-481b-bae4-7bafd683da67 dr2342 | https://www.gbif.org/dataset/11ba0f8a-b084-47f6-b711-9f995c6f8e79

Please delete the AVRO for this DR, not the collectory entries.

[cha801p]

Ticket Update: September 13, 2023 (4:30 PM)

Issue Resolved: Soft Deletion of Fossil Data Resources

Solution: We have successfully performed a soft delete operation on the Avro files for the following datasets: dr2290, dr2289, and dr2342. Soft delete means that we have removed the Avro files while retaining the UUIDs and the collectory records. (These datasets will not be included in the next index run)

Actions Taken: Executed the "Delete_dataset_dag" with the following parameters: "datasetIds": dr2290, dr2289, dr2342 "remove_records_in_solr": true "remove_records_in_es": false "delete_avro_files": true "retain_dwca": true "retain_uuid": true

Validation: I have confirmed the following:

• [x] UUIDs have been preserved.
• [x] Avro files have been successfully deleted from the S3 storage.
• [x] This action should not cause any issues in the upcoming indexing process.

[peggynewman]

Removed from GBIF, we can close this when we check the index and let Cam/Ely know we've done it

[peggynewman]

Done and done. Just a note for soft deleting, if anyone is coming back to this ticket later. The "Delete from GBIF" button in Collectory won't appear if the license on the data isn't compatible with GBIF. You can quickly edit the license and then the delete button will come back, and then you can use it.

[peggynewman]

PS needed to also delete the dwcas from dwca-imports/, because even re-interpretation reads the list of datasets to be loaded by reading this directory

[cha801p]

Done Re-ran Delete_dataset_dag with following parameters: "datasetIds": dr2290, dr2289, dr2342 "remove_records_in_solr": true "remove_records_in_es": false "delete_avro_files": true "retain_dwca": false "retain_uuid": true