search

AusClimateService / bias-correction-data-release

Tracking of bias corrected data release stages
3 stars 0 forks source link

missing data in ssp126 #8

Open matear opened 1 month ago

matear commented 1 month ago

Missing data in the following files were found. The ssp370 case was identified before but several ssp126 runs are missing data. The script lists missing days in the catalogue

################################# Running with index: 1 Zarr: AGCD-05i_BOM_ACCESS-CM2_ssp126_r4i1p1f1_BARPA-R_v1-r1_day /scratch/xv83/rxm599/nobiascorAGCD-05i_BOM_ACCESS-CM2_ssp126_r4i1p1f1_BARPA-R_v1-r1_day.zarr pr sfcWindmax tasmax hursmin hursmax tasmin 1224452 1224452 1224452 0 0 1224452 1224452 1224452 1224452 0 0 1224452 Indices of NaN values in : ['2049-07-01T12:00:00.000000000' '2049-07-02T12:00:00.000000000'] Indices of NaN values in : ['2049-07-01T12:00:00.000000000' '2049-07-02T12:00:00.000000000'] Indices of NaN values in : ['2049-07-01T12:00:00.000000000' '2049-07-02T12:00:00.000000000'] Indices of NaN values in : ['2049-07-01T12:00:00.000000000' '2049-07-02T12:00:00.000000000'] Indices of NaN values in : [] Indices of NaN values in : []

Running with index: 4 Zarr: AGCD-05i_BOM_ACCESS-ESM1-5_ssp126_r6i1p1f1_BARPA-R_v1-r1_day /scratch/xv83/rxm599/nobiascorAGCD-05i_BOM_ACCESS-ESM1-5_ssp126_r6i1p1f1_BARPA-R_v1-r1_day.zarr pr sfcWindmax tasmax hursmin hursmax tasmin 2448904 2448904 2448904 0 0 2448904 2448904 2448904 2448904 0 0 2448904 Indices of NaN values in : ['2073-09-09T12:00:00.000000000' '2073-09-10T12:00:00.000000000' '2073-10-16T12:00:00.000000000' '2073-10-17T12:00:00.000000000'] Indices of NaN values in : ['2073-09-09T12:00:00.000000000' '2073-09-10T12:00:00.000000000' '2073-10-16T12:00:00.000000000' '2073-10-17T12:00:00.000000000'] Indices of NaN values in : ['2073-09-09T12:00:00.000000000' '2073-09-10T12:00:00.000000000' '2073-10-16T12:00:00.000000000' '2073-10-17T12:00:00.000000000'] Indices of NaN values in : ['2073-09-09T12:00:00.000000000' '2073-09-10T12:00:00.000000000' '2073-10-16T12:00:00.000000000' '2073-10-17T12:00:00.000000000']

Running with index: 5 Zarr: AGCD-05i_BOM_ACCESS-ESM1-5_ssp370_r6i1p1f1_BARPA-R_v1-r1_day /scratch/xv83/rxm599/nobiascorAGCD-05i_BOM_ACCESS-ESM1-5_ssp370_r6i1p1f1_BARPA-R_v1-r1_day.zarr pr sfcWindmax tasmax hursmin hursmax tasmin 1224452 1224452 1224452 0 0 1224452 1224452 1224452 1224452 0 0 1224452 Indices of NaN values in : ['2029-04-21T12:00:00.000000000' '2029-04-22T12:00:00.000000000'] Indices of NaN values in : ['2029-04-21T12:00:00.000000000' '2029-04-22T12:00:00.000000000'] Indices of NaN values in : ['2029-04-21T12:00:00.000000000' '2029-04-22T12:00:00.000000000'] Indices of NaN values in : ['2029-04-21T12:00:00.000000000' '2029-04-22T12:00:00.000000000'] Indices of NaN values in : [] Indices of NaN values in : []

DamienIrving commented 1 month ago

Hi, @matear. It looks like those dates and times of missing data that you've listed match up exactly with the those listed in the "missing data" column of the ACCESS-CM2 and ACCESS-ESM1-5 data tables in the README of this repo: https://github.com/AusClimateService/bias-correction-data-release?tab=readme-ov-file#access-cm2 https://github.com/AusClimateService/bias-correction-data-release?tab=readme-ov-file#access-esm1-5

The problem is with the original BARPA data files - according to the known issues page for the BARPA data this problem is fixed, in that they've masked out the days that had erroneous data. They won't be re-running BARPA to generate new data for those days.