2023-2024 Data Mobilisation - DESS Kangaroo Island

[rosemaryjoconnor]

1998, Dr Richard Glatz (Chief Editor – Austral Entomology) started an insect collection focused on Kangaroo Island (KI; South Australia) and this collection now contains ≈70,000 specimens. The collection is located on KI and contains a range of undescribed species, many new records for the island, and thus far, paratypes for seven species described from KI.

Currently, the collection database consists of ≈16,000 records.

[rosemaryjoconnor]

Initial analysis done. GPS Data is a challenge. User emailed with issues and solutions. Full dataset is around 16.5K records.

[rosemaryjoconnor]

Update from RG @17/11/2023: Away until 23/11/2023 - will contact early December.

Briefly, so far I have looked through through the records and just corrected (in my database) any of the GPS data that were clearly wrong (e.g. in the ocean). Am planning to update the records (mainly classifications) in the first few months next year….up until the point you need the data resubmitted.

[rosemaryjoconnor]

dr21992 - test 07/12/2023

• RG has updated database structure, fixed GPS data.
• RG will review current dr21992 for any issues
• Aim to catch up late January
• Final dataset in April

I have suggested another interim dataset now for loading due to the changes.

[rosemaryjoconnor]

13/12/2023 New dataset provided.

[rosemaryjoconnor]

14/12/2023

• [x] New dataset loaded successfully
• [x] No records need to be dropped, all lat/lon issues are able to be fixed in the code.
• [x] Notebook converted to Python script, committed in GitHub - databox

[rosemaryjoconnor]

19/12/2024

• [x] RG email last set of updates sent very useful. He was aware of some of the issues.
• [x] RG to contact early 2024

[rosemaryjoconnor]

05/02/2024 - Mtg Dataset looking good.

• [x] RG had a few questions. Need to check:
• [x] Names matching
• [x] Asked for change in Catalogue # text based on date for post-fire
• [x] Record 8438 - check lat conversion
• [x] Records 2865->2867 need catalogNumber fixes (data issue)

[rosemaryjoconnor]

07/02/2024

• [x] Names matching fix from Cam
• [x] Code completed

Adding Kingdom (ie Animalia) into the kingdom field will improve the match and also help with the homonym thing. Adding sp. into the species field will help further. What will remove it entirely is adding author into the genus field as Heteronyx Guérin-Méneville, 1831 or adding into the authority field Guérin-Méneville, 1831 - doing authors is a lot of work, however.

[rosemaryjoconnor]

01/03/2024

• [x] Added Kingdom to dataset
• [x] Loaded to databox
• [x] Data checked

06/03/2024

• [x] Emailed Richard to update re names matching updates

[rosemaryjoconnor]

03/04/2024 Emailed RG for update and metadata request

To Do:

• [x] Test - Add metadata

[rosemaryjoconnor]

17/04/2024

To do:

• [x] Test - load final dataset - dr21992
• [x] Prod - Create DR in collectory - dr26187
• [x] Prod - Metadata

[rosemaryjoconnor]

31/05/2024

• [x] Emailed re final datasest

To do:

• [ ] Process final dataset
• [ ] Prod - Data Load

[rosemaryjoconnor]

06/06/2024

• RG has some questions re some filtered records - SS sorted that for me
• Final dataset for mobilisation to arrive next week

[rosemaryjoconnor]

19/06/2024

• Final dataset live in production 16,474 records

https://collections.ala.org.au/public/show/dr26187

[rosemaryjoconnor]

21/06/2024

• Data loaded in prod had longitude problems. The source data didn't have quite as many idiosyncracies as previous versions so the lat/lon fixes weren't quite correct. All fixed now in databox.
• The data loaded in prod was not the final dataset it seems.
• [x] Process new dataset
• [x] Load and test in databox
• [x] Load Production
• [x] SOLR Index complete

[rosemaryjoconnor]

26/06/2024

Data live in Prod: https://collections.ala.org.au/public/show/dr26187