Collection of datasets not in WOD

Thomas-Moore-Creative commented 1 year ago

Plan is to apply the IQuOD QC to the "boutique" datasets we are ingesting that don't already have QC applied

Note that IQuOD QC has not been applied to any of these datasets yet. If QC flags are available in the dataset, it is noted and which flags to include for @ChrisC28 and @Thomas-Moore-Creative mapping process

Datasets list:

[x] AIMS csv (no QC) - converted to NETCDF format 14/6/2023, no QC flags and only a date (no time stamp).
[x] AIMS netCDF (with QC) - available with QC flags in NETCDF format, use flags 1, 2 and 5 in the PARAMETER_quality_control variables.
[x] RAN CTD in netCDF (with QC) - available in the WOD/IQuOD. To test CARS with/without these data, look for 'Access_no' in the WOD netcdf files and remove profiles with 0104412. Suggest using Originator QC flags for these data (flags 1, 2 and 5), but if using WOD flags, use flag 0.
[ ] Wakmatha pCO2 netCDF (QC?)
[x] MNF CTD data extracted by @ChrisC28 - located in /tube1/cha674/CARS_2022_ancillary_data/MNF - Chris has started a reader.
[ ] ANMN moorings netCDF via the THREDDS server or Amazon cloud with QC
[ ] FAIMS temperature loggers from IMOS. Available at the oa-datalib/observations/FAIMMS
[ ] Glider data from IMOS. Available at the oa-datalib/observations/glider_nc
[ ] TrackObs SST available on AODN (source AIMS), contact is Helen Beggs https://thredds.aodn.org.au/thredds/catalog/IMOS/SRS/SST/catalog.html
[ ] Data from Nicole Jones, NWS moorings data. Potential for opening discussions with Industry to see if they will share their data. WAMSI have a project working with industry to contribute their data to a modelling project, but keep the data encrypted. This might be an idea for us to use to enable working with industry. SEAF. Cockburn Sound is the focus.
[ ] Spencer Gulf data from SARDI, Mark Doubell is the contact

Need to figure out the application of the IQuOD QC routines.

BecCowley commented 1 year ago

@ChrisC28 I have created individual NC files from the NCMI datatrawler for CTD data. Same format as the AIMS files. They are located in /oa-decadal-climate/work/observations/CARSv2_ancillary/MNF_GBR Dates covered from early 1980s to 2022.

ChrisC28 commented 1 year ago

GBR_case_study_mapped_clim_2005_2022 pfl xbt ctd WOD_and_AIMS Difference over the GBR region when including the AIMS+MNF data. Note! I haven't tuned or added advection to the mapping.

Thomas-Moore-Creative commented 1 year ago

@ChrisC28 - for advection BRAN2020 U&V is here > https://github.com/CARSv2/cars-v2/issues/13#issuecomment-1592316491

BecCowley commented 1 year ago

@ChrisC28, the AIMS GBR marine park project has just released some of the CTD profiles through AODN. There are files in here that are the same as the csv files. Except, a check of one of them (YON022) showed that the data is different. It looks like it's been through some sort of calibration/QC. But there are no flags. Anyway, I've put it in another folder 'AIMS_viaAODN'. Probably would use this over the CSV converted versions in the AIMS/NC2. There are 1608 files, compared with 2396 csv files, so not all that you downloaded are in the new format.

I will ask some questions about what has been done to the data.

CARSv2 / cars-v2