Format and index CMIP6 BCCAQ2 data

[corviday]

• [x] Remove test CMIP6 datasets from pcic_meta
• [x] convert exemplar datasets (ACCESS-CM and CanESM5) to netcdf4
• [x] update metadata on exemplar datasets
• [x] index exemplar datasets
• [x] associate exemplar datasets to ensemble
• [x] view exemplar datasets in demo portal, make any changes necessary

[corviday]

In order to not slam the backup system, I am doing data one model/folder at a time. This checklist lets me keep track of where I am on that.

ACCESS-CM2

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

ACCESS-ESM1-5

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

BCC-CSM2-MR

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

CANESM5 R1

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

CanESM5 other runs

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

CMCC-ESM2

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

CNRM-CM6-1

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

CNRM-ESM2-1

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

EC-Earth3

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

EC-Earth3-Veg

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

FGOALS-g3

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

GFDL-ESM4

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

HadGEM3-GC31-LL

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

INM-CM4-8

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles
• [x] update time units

INM-CM5-0

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles
• [x] update time units

IPSL-CM6A-LR

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

KACE-1-0-G

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

KIOST-ESM

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

MIROC6

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles
• [x] update time units

MIROC-ES2L

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles
• [x] update time units

MPI-ESM1-2-HR

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles
• [x] update time units

MPI-ESM1-2-LR

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles
• [x] update time units

MRI-ESM2-0

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

NorESM2-LM

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

NorESM2-MM

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

TaiESM1

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

UKESM1-0-LL

• [x] convert to netCDF4
• [x] update metadata
• [x] normalize timestamps
• [x] index
• [x] associate to ensembles

[corviday]

After all data is processed:

• [x] rsync to compute canada
• [x] reboot the ncWMS proxy server

[corviday]

• [x] process anomalous dataset, when fixed