Set up iNaturalist extracts

[peggynewman]

Set up iNaturalist exports in S3 using the December export (see directory in S3) as previously done for Tasmania. We need these for Tas, Qld and National.

Let me know when they're ready and I will provide contact names for each.

Share the notebook that does this work on a private repo in GH, but at the same time we need to ensure no sensitive data is stored on GH. We can discuss this. Perhaps we should just maintain the notebook in S3.

[peggynewman]

Closing this, superceded by https://github.com/AtlasOfLivingAustralia/preingestion/issues/105

[peggynewman]

Reopening this issue, as it belongs in the data-management repo, and closing the issue in preingestion instead.

@cha801p I've created a notebook to do this, which generates a report on the counts that need to be sent to people who request access to this data

https://github.com/AtlasOfLivingAustralia/databox/tree/master/inaturalist-sensitive-exports See https://confluence.csiro.au/display/ALASD/iNaturalist+Sensitive+Observation+Data

Still some work for you to do to empty private_ fields and create individual archives for all states and territories, and load them into S3.

[peggynewman]

Notebook still to be pushed to GH

[peggynewman]

File loading into S3 now - these exports are to be set up for 202306

[cha801p]

Ticket Update: 4 September, 2023 (4:30 PM)

Issue Identified: Set up iNat Extracts

Resolution: State-wise csv and zip files created and uploaded on s3 bucket

S3 location: /202306/output/

Next step: Waiting for the next update from Peggy after communicating with Cam

[cha801p]

Ticket Update: 4 October, 2023 (5:30 PM)

Issue Identified: Set up iNat Extracts

Current Status: State-wise CSV and zip files were created and uploaded to the S3 bucket

S3 location: /202306/output/

Steps Taken:

1. Received a follow-up email from the Tasmania data requester on 3rd October, 2023
2. Email draft sent to Cam, Peggy, and Mahmoud with Data download link

Next step:

• [x] Waiting for the next update from Peggy and Cam

[cha801p]

Ticket Update: 14 December, 2023 (3:00 PM)

Issue Identified: Set up iNat Extracts

Current Status: State-wise CSV and zip files were created and uploaded to the S3 bucket

S3 location: /202306/output/

Steps Taken: Peggy has emailed the data requester

Next step: Waiting for an update from Cam/Peggy/Others

[peggynewman]

Waiting on me to discuss this with Cam

[peggynewman]

Superceded by https://github.com/AtlasOfLivingAustralia/data-management/issues/1008