search

xCDAT / xcdat

An extension of xarray for climate data analysis on structured grids.
https://xcdat.readthedocs.io/en/latest/
Apache License 2.0
113 stars 12 forks source link

[Feature]: Improve `KeyError` message when `cf_xarray` can't find CF attributes for coordinates with `regrid2` #447

Closed lee1043 closed 1 year ago

lee1043 commented 1 year ago

What happened?

Spatial horizontal interpolation error for high resolution regional dataset

What did you expect to happen?

I am interpolating high resolution regional data (over CONUS) to global grid, expecting data regridded with keeping missing area (thus, in regrided data, it will have data only over CONUS).

Minimal Complete Verifiable Example

>>> import xcdat as xc
>>> ds= xc.open_mfdataset("pr.historical.ACCESS-CM2.r1i1p1f1.mon.198101-200512.AC.v20230404.nc")
/global/homes/l/lee1043/.conda/envs/pmp_devel_20230213/lib/python3.9/site-packages/pandas/core/arrays/datetimelike.py:1384: CFWarning: this date/calendar/year zero convention is not supported by CF
  res_values = op(self.astype("O"), np.asarray(other))
>>> t_grid = xc.create_uniform_grid(-88.875, 88.625, 2.5, 0, 357.5, 2.5)
>>> ds_regrid = ds.regridder.horizontal('pr_tavg', t_grid, tool='regrid2')

Relevant log output

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/global/homes/l/lee1043/.conda/envs/pmp_devel_20230213/lib/python3.9/site-packages/xcdat/regridder/accessor.py", line 286, in horizontal
    output_ds = regridder.horizontal(data_var, self._ds)
  File "/global/homes/l/lee1043/.conda/envs/pmp_devel_20230213/lib/python3.9/site-packages/xcdat/regridder/regrid2.py", line 135, in horizontal
    output_axis_sizes = self._output_axis_sizes(input_data_var)
  File "/global/homes/l/lee1043/.conda/envs/pmp_devel_20230213/lib/python3.9/site-packages/xcdat/regridder/regrid2.py", line 175, in _output_axis_sizes
    axis_name = axis_name_map[standard_name]
KeyError: 'lat'

Anything else we need to know?

Example input file for reproducing the error: pr.historical.ACCESS-CM2.r1i1p1f1.mon.198101-200512.AC.v20230404.nc.zip

Environment

xcdat 0.4.0 pmp 3.0.1

pochedls commented 1 year ago

@taylor13 might have thoughts on this since you are using regrid2.

I think I recently had this issue recently with ESMF (curvilinear to rectilinear). I can't remember exactly what happened, but bilinear/nearest neighbor gave bad values (maybe zeros or another constant outside the domain in which there was data in the source grid). I didn't raise an issue, because I wasn't sure ESMF should know what to do about that.

lee1043 commented 1 year ago

For this particular data I noticed

ds['pr_tavg'].cf.axes returns only {'T': ['time']}, instead {'X': ['lon'], 'Y': ['lat'], 'T': ['time']} that I think it supposed to be like.

I was trying to override by manually adding the correct keys, but it seems it is not allowed. @tomvothecoder do you have any idea how best this info could be added?

ds["pr_tavg"].cf.axes = {'X': ['lon'], 'Y': ['lat'], 'T': ['time']}
AttributeError: can't set attribute 'axes'
pochedls commented 1 year ago

@lee1043 - I'm curious – if you run ncdump -h pr.historical.ACCESS-CM2.r1i1p1f1.mon.198101-200512.AC.v20230404.nc - is there something unique about the metadata that might indicate why the lat/lon axes aren't being recognized?

lee1043 commented 1 year ago

@pochedls thank you for the suggestion. I cannot find anything specific. 

netcdf pr.historical.ACCESS-CM2.r1i1p1f1.mon.198101-200512.AC.v20230404 {
dimensions:
    time = UNLIMITED ; // (12 currently)
    lat = 474 ;
    lon = 944 ;
    bnds = 2 ;
variables:
    float lat(lat) ;
        lat:_FillValue = NaNf ;
        lat:units = "degreesN" ;
        lat:long_name = "lat" ;
        lat:bounds = "lat_bnds" ;
    float lon(lon) ;
        lon:_FillValue = NaNf ;
        lon:units = "degreesE" ;
        lon:long_name = "lon" ;
        lon:bounds = "lon_bnds" ;
    float lon_bnds(lon, bnds) ;
        lon_bnds:_FillValue = NaNf ;
        lon_bnds:xcdat_bounds = "True" ;
    float lat_bnds(lat, bnds) ;
        lat_bnds:_FillValue = NaNf ;
        lat_bnds:xcdat_bounds = "True" ;
    double time(time) ;
        time:_FillValue = NaN ;
        time:long_name = "time" ;
        time:bounds = "time_bnds" ;
        time:axis = "T" ;
        time:standard_name = "time" ;
        time:units = "days since 1900-01-01" ;
        time:calendar = "standard" ;
    double pr_tavg(time, lat, lon) ;
        pr_tavg:_FillValue = NaN ;
        pr_tavg:units = "kg m-2 s-1" ;
        pr_tavg:history = "2019-11-09T02:13:28Z altered by CMOR: replaced missing value flag (-1.07374e+09) with standard missing value (1e+20). 2021/08/04 01:35:23 pyao: ncunary -ignore_bounds -var pr -start 121,87,1 -count 47,31,: -out ACCESS-CM2/1x1/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.1950-2014.tmpcropspace.nc /nobackupp10/dwpierc2/CMIP6/archive/ACCESS-CM2/historical/r1i1p1f1/pr/pr_day_ACCESS-CM2_historical_r1i1p1f1_gn_18500101-18991231.nc /nobackupp10/dwpierc2/CMIP6/archive/ACCESS-CM2/historical/r1i1p1f1/pr/pr_day_ACCESS-CM2_historical_r1i1p1f1_gn_19000101-19491231.nc /nobackupp10/dwpierc2/CMIP6/archive/ACCESS-CM2/historical/r1i1p1f1/pr/pr_day_ACCESS-CM2_historical_r1i1p1f1_gn_19500101-19991231.nc /nobackupp10/dwpierc2/CMIP6/archive/ACCESS-CM2/historical/r1i1p1f1/pr/pr_day_ACCESS-CM2_historical_r1i1p1f1_gn_20000101-20141231.nc 2021/08/04 01:45:33 pyao: ncunary -ignore_bounds -lt 0.0 -setto 0.0 -out ACCESS-CM2/r1i1p1f1.historical.tzp.pr.historical.1950-2014.nc ACCESS-CM2/1x1/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.1950-2014.tmpcropspace.nc 2021/08/04 01:50:13 pyao: ncunary -ignore_bounds -start 1,1,36525 -count :,:,23741 -out ACCESS-CM2/1x1/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.1950-2014.crop.nc ACCESS-CM2/1x1/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.1950-2014.tmpcropspace.n #### 2022-03-30 20:16:51 /home6/dwpierc2/src/mine/bcca2/version_61/loca_make_4_8_alt_file -years_hist 1950 2014 -fname_hist ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.1950-2014.crop.stdcal.nc -years_ssp 2015 2039 -stdout_fname /nobackupp10/dwpierc2/LOCAds2014/LOCA2_2021-08-27/all_data_proc/run/outfiles.13138982.pbspl1.nas.nasa.gov.2022-03-30_20-16/loca_make_4_8_alt_file.0p5x0p5.ACCESS-CM2.pr.historical.stdout -fname_ssp ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp245/pr/pr.ACCESS-CM2.ssp245.2015-2044.crop.stdcal.nc,../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp370/pr/pr.ACCESS-CM2.ssp370.2015-2044.crop.stdcal.nc,../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp585/pr/pr.ACCESS-CM2.ssp585.2015-2044.crop.stdcal.nc -fname_out ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.1950-2014.crop.stdcal.4_8_file.n #### 2022-03-30 20:16:55 /home6/dwpierc2/src/mine/bcca2/version_61/loca_bias_correct -presrat -iterative_schedule 8 -maxfrac_precond_limit 1.0 -seed 315259581 -train_period 1950-2014 -nthreads 24 -stdout_fname /nobackupp10/dwpierc2/LOCAds2014/LOCA2_2021-08-27/all_data_proc/run/outfiles.13138982.pbspl1.nas.nasa.gov.2022-03-30_20-16/loca_bias_correct.0p5x0p5.ACCESS-CM2.pr.r1i1p1f1.historical.stdout -vartype precip@kg.m-2.s-1 -prune_cdf 2 -fname_obs ../../Models/ACCESS-CM2/Livneh.pr.on_ACCESS-CM2_grid.1950-2014.nc -fname_gcm_train =ENS+:r1i1p1f1:r2i1p1f1:r3i1p1f1:=../../Models/ACCESS-CM2/0p5x0p5/=ENS=/historical/pr/pr.ACCESS-CM2.historical.1950-2014.crop.stdcal.nc -gcm_data_to_bc =ENS+:r1i1p1f1:r2i1p1f1:r3i1p1f1:=../../Models/ACCESS-CM2/0p5x0p5/=ENS=/historical/pr/pr.ACCESS-CM2.historical.1950-2014.crop.stdcal.4_8_file.nc -out_years 1950 2014 -fname_bc_out =ENS+:r1i1p1f1:r2i1p1f1:r3i1p1f1:=../../Models/ACCESS-CM2/0p5x0p5/=ENS=/historical/pr/pr.ACCESS-CM2.historical.=ENS=.1950-2014.crop.stdcal.bc.n #### 2022-03-30 22:48:22 /home6/dwpierc2/src/mine/bcca2/version_61/loca_make_4_8_alt_file -years_hist 1950 2014 -fname_hist ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.nc -years_ssp 2015 2039 -stdout_fname /nobackupp10/dwpierc2/LOCAds2014/LOCA2_2021-08-27/all_data_proc/run/outfiles.13138982.pbspl1.nas.nasa.gov.2022-03-30_20-16/loca_make_4_8_alt_file.0p5x0p5.ACCESS-CM2.pr.historical.bc.stdout -fname_ssp ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp245/pr/pr.ACCESS-CM2.ssp245.r1i1p1f1.2015-2100.crop.stdcal.bc.nc,../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp370/pr/pr.ACCESS-CM2.ssp370.r1i1p1f1.2015-2100.crop.stdcal.bc.nc,../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp585/pr/pr.ACCESS-CM2.ssp585.r1i1p1f1.2015-2100.crop.stdcal.bc.nc -fname_out ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.4_8_file.bc.n #### 2022-03-30 22:48:31 /home6/dwpierc2/src/mine/bcca2/version_61/loca_spec_reshape -hist_period 1950-2014 -vartype precip@kg.m-2.s-1 -nthreads 28 -modthresh 1.157e-05 -stdout_fname /nobackupp10/dwpierc2/LOCAds2014/LOCA2_2021-08-27/all_data_proc/run/outfiles.13138982.pbspl1.nas.nasa.gov.2022-03-30_20-16/loca_spec_reshape.0p5x0p5.ACCESS-CM2.pr.r1i1p1f1.historical.stdout -nlags 1020 -fname_obs ../../Models/ACCESS-CM2/Livneh.pr.on_ACCESS-CM2_grid.1950-2014.nc -fname_gcm_hist ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.4_8_file.bc.nc -years_gcm_in 1950 2014 -fname_gcm_in ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.4_8_file.bc.nc -fname_out ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.n #### 2022-03-31 06:39:32 /home6/dwpierc2/src/mine/bcca2/version_61/loca_downscale -smooth_edges -vartype precip@kg.m-2.s-1 -nthreads 24 -incore -phase 2 -targetradius 4 -output_copacetic_fname ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.nc.checkedOK -outmask output_mask_0p5x0p5.cd.nc -preproc_obs ../../Models/ACCESS-CM2/Livneh.pr.1950-2014.crop.0p5x0p5.preproc_obs_for_ACCESS-CM2.nc -years_gcm_hist 1950 2014 -fname_gcm_hist ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.nc -years_gcm_in 1950 2014 -fname_gcm_in ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.nc -fname_out ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.n #### 2022-04-01 09:46:19 /home6/dwpierc2/src/mine/bcca2/version_61/loca_make_4_8_alt_file -years_hist 1950 2014 -fname_hist ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.nc -years_ssp 2015 2039 -stdout_fname /nobackupp10/dwpierc2/LOCAds2014/LOCA2_2021-08-27/all_data_proc/run/outfiles.13147680.pbspl1.nas.nasa.gov.2022-04-01_09-46/loca_make_4_8_alt_file.0p0625deg.ACCESS-CM2.pr.historical.stdout -fname_ssp ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp245/pr/pr.ACCESS-CM2.ssp245.r1i1p1f1.2015-2044.crop.stdcal.bc.srs.0p5x0p5.nc,../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp370/pr/pr.ACCESS-CM2.ssp370.r1i1p1f1.2015-2044.crop.stdcal.bc.srs.0p5x0p5.nc,../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp585/pr/pr.ACCESS-CM2.ssp585.r1i1p1f1.2015-2044.crop.stdcal.bc.srs.0p5x0p5.nc -fname_out ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.4_8_file.n #### 2022-04-01 09:47:18 /home6/dwpierc2/src/mine/bcca2/version_61/loca_bias_correct -presrat -iterative_schedule 8 -maxfrac_precond_limit 1.0 -seed 198676219 -train_period 1950-2014 -nthreads 24 -stdout_fname /nobackupp10/dwpierc2/LOCAds2014/LOCA2_2021-08-27/all_data_proc/run/outfiles.13147680.pbspl1.nas.nasa.gov.2022-04-01_09-46/loca_bias_correct.0p0625deg.ACCESS-CM2.pr.r1i1p1f1.historical.stdout -outmask output_mask_0p5x0p5.cd.nc -vartype precip@kg.m-2.s-1 -prune_cdf 2 -fname_obs ../../training_data/LOCA2_training_2022-03-30.precip.kg.m-2.s-1.1950-2014.v2.0p5x0p5.nc -fname_gcm_train =ENS+:r1i1p1f1:r2i1p1f1:r3i1p1f1:=../../Models/ACCESS-CM2/0p5x0p5/=ENS=/historical/pr/pr.ACCESS-CM2.historical.=ENS=.1950-2014.crop.stdcal.bc.srs.0p5x0p5.nc -gcm_data_to_bc =ENS+:r1i1p1f1:r2i1p1f1:r3i1p1f1:=../../Models/ACCESS-CM2/0p5x0p5/=ENS=/historical/pr/pr.ACCESS-CM2.historical.=ENS=.1950-2014.crop.stdcal.bc.srs.0p5x0p5.4_8_file.nc -out_years 1950 2014 -fname_bc_out =ENS+:r1i1p1f1:r2i1p1f1:r3i1p1f1:=../../Models/ACCESS-CM2/0p5x0p5/=ENS=/historical/pr/pr.ACCESS-CM2.historical.=ENS=.2015-2014.crop.stdcal.bc.srs.0p5x0p5.bc.n #### 2022-04-02 06:47:48 /home6/dwpierc2/src/mine/bcca2/version_61/loca_make_4_8_alt_file -years_hist 1950 2014 -fname_hist ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.2015-2014.crop.stdcal.bc.srs.0p5x0p5.bc.nc -years_ssp 2015 2039 -stdout_fname /nobackupp10/dwpierc2/LOCAds2014/LOCA2_2021-08-27/all_data_proc/run/outfiles.13151576.pbspl1.nas.nasa.gov.2022-04-02_01-42/loca_make_4_8_alt_file.0p0625deg.ACCESS-CM2.pr.historical.bc.stdout -fname_ssp ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp245/pr/pr.ACCESS-CM2.ssp245.r1i1p1f1.2015-2100.crop.stdcal.bc.srs.0p5x0p5.bc.nc,../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp370/pr/pr.ACCESS-CM2.ssp370.r1i1p1f1.2015-2100.crop.stdcal.bc.srs.0p5x0p5.bc.nc,../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp585/pr/pr.ACCESS-CM2.ssp585.r1i1p1f1.2015-2100.crop.stdcal.bc.srs.0p5x0p5.bc.nc -fname_out ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.2015-2014.crop.stdcal.bc.srs.0p5x0p5.bc.4_8_file.n #### 2022-04-02 06:48:20 /home6/dwpierc2/src/mine/bcca2/version_61/loca_spec_reshape -hist_period 1950-2014 -vartype precip@kg.m-2.s-1 -nthreads 28 -modthresh 1.157e-05 -stdout_fname /nobackupp10/dwpierc2/LOCAds2014/LOCA2_2021-08-27/all_data_proc/run/outfiles.13151576.pbspl1.nas.nasa.gov.2022-04-02_01-42/loca_spec_reshape.0p0625deg.ACCESS-CM2.pr.r1i1p1f1.historical.stdout -nlags 1020 -fname_obs ../../training_data/LOCA2_training_2022-03-30.precip.kg.m-2.s-1.1950-2014.v2.0p5x0p5.nc -fname_gcm_hist ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.2015-2014.crop.stdcal.bc.srs.0p5x0p5.bc.4_8_file.nc -years_gcm_in 1950 2014 -fname_gcm_in ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.2015-2014.crop.stdcal.bc.srs.0p5x0p5.bc.4_8_file.nc -fname_out ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.bc.srs.n #### 2022-04-02 18:23:07 /home6/dwpierc2/src/mine/bcca2/version_61/loca_downscale -smooth_edges -vartype precip@kg.m-2.s-1 -nthreads 24 -incore -stdout_fname ../../Models/ACCESS-CM2/0p0625deg/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.bc.srs.ds.nc.2022-04-02_18-23.stdout -phase 2 -targetradius 10 -output_copacetic_fname ../../Models/ACCESS-CM2/0p0625deg/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.bc.srs.ds.nc.checkedOK -preproc_obs ../../training_data/LOCA2_training_2022-03-30.precip.kg.m-2.s-1.1950-2014.v2.preproc_obs.nc -years_gcm_hist 1950 2014 -fname_gcm_hist ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.bc.srs.nc -years_gcm_in 1950 2014 -fname_gcm_in ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.bc.srs.nc -fname_out ../../Models/ACCESS-CM2/0p0625deg/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.bc.srs.ds.n #### 2022-05-08 17:34:50 /home6/dwpierc2/src/mine/bcca2/version_61/loca_post_ds_bc -nthreads 20 -fname_maxout_2d /nobackupp10/dwpierc2/LOCAds2014/LOCA2_2021-08-27/training_data/max_precip_mask.2022-05-08.nc -fname_no_mod_if_gt ../../Models/ACCESS-CM2/0p0625deg/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.bc.srs.ds.ptiles.p95.nc -bcwin_width 3 -vartype precip@kg.m-2.s-1 -valid_range_mod_predict_del 0.333 3.0 -valid_range_mod_error 0.5 2.0 -spat_smo_efwidth_deg 1.0 -fname_save_consts_out ../../Models/ACCESS-CM2/0p0625deg/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.bc.srs.ds.postdsbc.consts.nc -hist_period 1950-2014 -fut_periods 2015-2044,2045-2074,2075-2100 -fname_ensavg_hist_clim_n ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.1950-2014.crop.stdcal.monthly.clim.ENS_AVG.nc -fname_obs_fine group_obs_fine/precip:../../training_data/LOCA2_training_2022-03-30.precip.kg.m-2.s-1.1950-2014.v2.preproc_obs.nc -fname_gcm_hist_native ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.1950-2014.crop.stdcal.nc -fname_gcm_hist_fine ../../Models/ACCESS-CM2/0p0625deg/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.bc.srs.ds.nc -fname_gcm_fut_native ../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp245/pr/pr.ACCESS-CM2.ssp245.2015-2044.crop.stdcal.nc,../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp245/pr/pr.ACCESS-CM2.ssp245.2045-2074.crop.stdcal.nc,../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/ssp245/pr/pr.ACCESS-CM2.ssp245.2075-2100.crop.stdcal.nc -fname_gcm_fut_fine ../../Models/ACCESS-CM2/0p0625deg/r1i1p1f1/ssp245/pr/pr.ACCESS-CM2.ssp245.r1i1p1f1.2015-2044.crop.stdcal.bc.srs.0p5x0p5.bc.srs.ds.nc,../../Models/ACCESS-CM2/0p0625deg/r1i1p1f1/ssp245/pr/pr.ACCESS-CM2.ssp245.r1i1p1f1.2045-2074.crop.stdcal.bc.srs.0p5x0p5.bc.srs.ds.nc,../../Models/ACCESS-CM2/0p0625deg/r1i1p1f1/ssp245/pr/pr.ACCESS-CM2.ssp245.r1i1p1f1.2075-2100.crop.stdcal.bc.srs.0p5x0p5.bc.srs.ds.nc -fname_gcm_out_hist ../../Models/ACCESS-CM2/0p0625deg/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.bc.srs.ds.postdsbc.nc -fname_gcm_out_fut ../../Models/ACCESS-CM2/0p0625deg/r1i1p1f1/ssp245/pr/pr.ACCESS-CM2.ssp245.r1i1p1f1.2015-2044.crop.stdcal.bc.srs.0p5x0p5.bc.srs.ds.postdsbc.nc,../../Models/ACCESS-CM2/0p0625deg/r1i1p1f1/ssp245/pr/pr.ACCESS-CM2.ssp245.r1i1p1f1.2045-2074.crop.stdcal.bc.srs.0p5x0p5.bc.srs.ds.postdsbc.nc,../../Models/ACCESS-CM2/0p0625deg/r1i1p1f1/ssp245/pr/pr.ACCESS-CM2.ssp245.r1i1p1f1.2075-2100.crop.stdcal.bc.srs.0p5x0p5.bc.srs.ds.postdsbc.nc 2022/10/28 11:48:16 dwpierc2: nctavg -monthly -out ./ACCESS-CM2/0p0625deg/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.LOCA_16thdeg_v20220519.monthly.nc ./ACCESS-CM2/0p0625deg/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.LOCA_16thdeg_v20220519.nc" ;
        pr_tavg:long_name = "pr" ;
        pr_tavg:operation = "temporal_avg" ;
        pr_tavg:mode = "climatology" ;
        pr_tavg:freq = "month" ;
        pr_tavg:weighted = "True" ;

// global attributes:
        :history = "2019-11-09T02:13:29Z ; CMOR rewrote data to be consistent with CMIP6, CF-1.7 CMIP-6.2 and CF standards." ;
        :SIOCRD_netCDF_Version = 1.f ;
        :title = "LOCA statistically downscaled climate model data, David W. Pierce, Scripps Institution of Oceanography" ;
        :Conventions = "CF-1.7 CMIP-6.2" ;
        :activity_id = "CMIP" ;
        :branch_method = "standard" ;
        :branch_time_in_child = 0. ;
        :branch_time_in_parent = 0. ;
        :creation_date = "2019-11-09T02:13:29Z" ;
        :data_specs_version = "01.00.30" ;
        :experiment = "" ;
        :experiment_id = "historical" ;
        :external_variables = "areacella" ;
        :forcing_index = 1 ;
        :frequency = "day" ;
        :further_info_url = "https://furtherinfo.es-doc.org/CMIP6.CSIRO-ARCCSS.ACCESS-CM2.historical.none.r1i1p1f1" ;
        :grid = "native atmosphere N96 grid (144x192 latxlon)" ;
        :grid_label = "gn" ;
        :initialization_index = 1 ;
        :institution = "CSIRO (Commonwealth Scientific and Industrial Research Organisation, Aspendale, Victoria 3195, Australia), ARCCSS (Australian Research Council Centre of Excellence for Climate System Science)" ;
        :institution_id = "CSIRO-ARCCSS" ;
        :mip_era = "CMIP6" ;
        :nominal_resolution = "250 km" ;
        :notes = "Exp: CM2-historical; Local ID: bj594; Variable: pr ([\'fld_s05i216\'])" ;
        :parent_activity_id = "CMIP" ;
        :parent_experiment_id = "piControl" ;
        :parent_mip_era = "CMIP6" ;
        :parent_source_id = "ACCESS-CM2" ;
        :parent_time_units = "days since 0950-01-01" ;
        :parent_variant_label = "r1i1p1f1" ;
        :physics_index = 1 ;
        :product = "model-output" ;
        :realization_index = 1 ;
        :realm = "atmos" ;
        :run_variant = "forcing: GHG, Oz, SA, Sl, Vl, BC, OC, (GHG = CO2, N2O, CH4, CFC11, CFC12, CFC113, HCFC22, HFC125, HFC134a)" ;
        :source = "ACCESS-CM2 (2019): \naerosol: UKCA-GLOMAP-mode\natmos: MetUM-HadGEM3-GA7.1 (N96; 192 x 144 longitude/latitude; 85 levels; top level 85 km)\natmosChem: none\nland: CABLE2.5\nlandIce: none\nocean: ACCESS-OM2 (GFDL-MOM5, tripolar primarily 1deg; 360 x 300 longitude/latitude; 50 levels; top grid cell 0-10 m)\nocnBgchem: none\nseaIce: CICE5.1.2 (same grid as ocean)" ;
        :source_id = "ACCESS-CM2" ;
        :source_type = "AOGCM" ;
        :sub_experiment = "none" ;
        :sub_experiment_id = "none" ;
        :table_id = "day" ;
        :table_info = "Creation Date:(30 April 2019) MD5:e14f55f257cceafb2523e41244962371" ;
        :variable_id = "pr" ;
        :variant_label = "r1i1p1f1" ;
        :version = "v20191108" ;
        :cmor_version = "3.4.0" ;
        :tracking_id = "hdl:21.14100/9f20f980-3ddb-496f-a42c-1bce8a4de2d6" ;
        :license = "CMIP6 model data produced by CSIRO is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (https://creativecommons.org/licenses/).  Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment.  Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file).  The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ;
        :loca_version = "loca_downscale\n$Id: loca_downscale.F90,v 1.87 2022/02/13 20:31:33 dwpierc2 Exp $\n$Id: cdf_mod.F90,v 1.26 2022/01/07 05:38:20 dwpierc2 Exp $\n $Id: cdft_mod.F90,v 1.23 2021/12/24 17:25:32 dwpierc2 Exp $\n $Id: climstats_lib_mod.F90,v 1.35 2022/02/13 20:30:28 dwpierc2 Exp $\n $Id: climstats_type_mod.f90,v 1.11 2021/12/24 17:25:32 dwpierc2 Exp $\n $Id: edcdfm_type_mod.F90,v 1.6 2021/12/24 17:25:32 dwpierc2 Exp $\n $Id: edcdfm_mod.F90,v 1.47 2022/02/26 00:55:27 dwpierc2 Exp $\n $Id: globconst_mod.f90,v 1.24 2021/12/24 17:25:32 dwpierc2 Exp $\n $Id: image_pyramid_module.f90,v 1.6 2022/01/17 16:57:48 dwpierc2 Exp $\n $Id: loca_routines_module.F90,v 1.121 2022/02/13 20:32:58 dwpierc2 Exp $\n $Id: masklib_routines_module.F90,v 1.27 2021/12/24 17:25:32 dwpierc2 Exp $\n $Id: masklib_types_module.f90,v 1.12 2021/12/24 17:25:32 dwpierc2 Exp $\n $Id: misc_early_routines_module.F90,v 1.13 2021/12/24 17:25:32 dwpierc2 Exp $\n $Id: misc_late_routines_module.F90,v 1.10 2022/01/17 05:56:23 dwpierc2 Exp $\n $Id: ncio_mod.F90,v 1.77 2022/02/13 20:33:40 dwpierc2 Exp $\n $Id: qq_mod.F90,v 1.24 2021/12/24 17:25:32 dwpierc2 Exp $\n $Id: reshape_spectrum_mod.F90,v 1.18 2021/12/24 17:25:32 dwpierc2 Exp $\n $Id: cubic_spline_mod.F90,v 1.10 2021/11/22 04:58:57 pierce Exp $\n $Id: cubic_hermite_mod.F90,v 1.8 2021/11/22 04:58:57 pierce Exp $\n $Id: ez_netcdf_mod.F90,v 1.2 2021/11/11 17:41:28 pierce Exp $\n $Id: progress_meter_module.f90,v 1.10 2016/04/19 16:12:53 pierce Exp $\n $Id: wherenearest_module.f90,v 1.4 2013/12/28 17:53:50 pierce Exp $\n $Id: parse_args_module.f90,v 1.18 2021/11/19 03:49:20 pierce Exp $\n $Id: coarsen_array_mod.f90,v 1.4 2013/12/28 18:33:09 pierce Exp $\n $Id: strings_module.f90,v 1.6 2021/11/12 16:50:06 pierce Exp $\n $Id: crdnetcdf_mod.F90,v 1.14 2022/01/20 17:45:44 pierce Exp $\n $Id: misc_module.f90,v 1.5 2016/08/23 21:32:25 pierce Exp $\n $Id: heapsort_mod.f90,v 1.4 2021/11/11 17:41:28 pierce Exp $\n $Id: ooze_mod.f90,v 1.5 2013/12/28 17:53:50 pierce Exp $\n $Id: correlation_module.f90,v 1.4 2021/11/11 17:41:28 pierce Exp $\n $Id: check_vals_module.f90,v 1.7 2021/11/11 17:41:28 pierce Exp $\n $Id: text_utils_mod.f90,v 1.9 2021/11/20 17:47:38 pierce Exp $\n $Id: regridpack_mod.f90,v 1.5 2021/11/11 17:41:28 pierce Exp $\n $Id: interpolate_mod.F90,v 1.11 2021/11/12 16:50:06 pierce Exp $\n $Id: current_time_sec_mod.f90,v 1.3 2013/12/28 17:53:50 pierce Exp $\n $Id: quicksort_mod.f90,v 1.9 2021/11/12 16:50:06 pierce Exp $\n $Id: spectrum_mod.f90,v 1.6 2013/12/28 17:53:50 pierce Exp $\n $Id: bestfitline_mod.f90,v 1.5 2014/09/29 18:53:55 pierce Exp $\n $Id: gauss_dist_mod.f90,v 1.3 2021/11/11 17:41:28 pierce Exp $\n $Id: utCalendar2_cal_mod.f90,v 1.1 2014/06/08 22:24:57 pierce Exp pierce $\n $Id: distrib_pts_in_domain_mod.F90,v 1.3 2015/06/25 17:32:13 pierce Exp $\n$Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/loca_downscale.F90,v $\n$Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/cdf_mod.F90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/cdft_mod.F90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/climstats_lib_mod.F90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/climstats_type_mod.f90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/edcdfm_type_mod.F90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/edcdfm_mod.F90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/globconst_mod.f90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/image_pyramid_module.f90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/loca_routines_module.F90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/masklib_routines_module.F90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/masklib_types_module.f90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/misc_early_routines_module.F90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/misc_late_routines_module.F90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/ncio_mod.F90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/qq_mod.F90,v $\n $Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/reshape_spectrum_mod.F90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/cubic_spline_mod.F90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/cubic_hermite_mod.F90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/ez_netcdf_mod.F90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/progress_meter_module.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/wherenearest_module.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/parse_args_module.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/coarsen_array_mod.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/strings_module.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/crdnetcdf_mod.F90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/misc_module.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/heapsort_mod.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/ooze_mod.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/correlation_module.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/check_vals_module.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/text_utils_mod.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/regridpack_mod.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/interpolate_mod.F90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/current_time_sec_mod.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/quicksort_mod.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/spectrum_mod.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/bestfitline_mod.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/gauss_dist_mod.f90,v $\n $Source: /home/pierce/src/mine/calcalcs/calcalcs_1.2/RCS/utCalendar2_cal_mod.f90,v $\n $Source: /home/pierce/src/mine/f90_modules/gfortran/RCS/distrib_pts_in_domain_mod.F90,v $" ;
        :ID_loca_routines_module = "$Id: loca_routines_module.F90,v 1.121 2022/02/13 20:32:58 dwpierc2 Exp $" ;
        :SOURCE_loca_routines_module = "$Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/loca_routines_module.F90,v $" ;
        :opts_var_type = "precip@kg.m-2.s-1" ;
        :opts_dscale_using_anoms = "FALSE" ;
        :opts_do_exclude = "FALSE" ;
        :opts_do_selfcrossval = "FALSE" ;
        :opts_pateval = "RMSE" ;
        :opts_n_patterns = 30 ;
        :opts_time_window_days = 45 ;
        :opts_do_min_val_out = "FALSE" ;
        :opts_do_max_val_out = "FALSE" ;
        :opts_do_min_val_in = "FALSE" ;
        :opts_dsclim_method = "DSMETHOD_DSCLIM (downscale change in climatology, as in Pierce and Cayan 2013)" ;
        :opts_gcm_preserve_zero = -1.e+35f ;
        :opts_ds_preserve_zero = -1.e+35f ;
        :opts_do_transform = "FALSE" ;
        :opts_target_radius = 10 ;
        :opts_incore = "TRUE" ;
        :floating_anaday_windows = "FALSE" ;
        :opts_correlation_mask_threshold = 1.e+30f ;
        :opts_mempack = "FALSE" ;
        :opts_smooth_edges = "TRUE" ;
        :opts_scalefact_max = 2.f ;
        :use_bilinear_interpol = "FALSE" ;
        :masklib_n_anapool_pts = 320 ;
        :masklib_nseasons = 4 ;
        :masklib_anapool_locs_coarse_grid_X = 27, 79, 101, 56, 5, 15, 87, 39, 50, 68, 91, 23, 9, 54, 62, 53, 98, 57, 116, 34, 15, 103, 72, 27, 101, 103, 83, 81, 110, 116, 63, 79, 103, 43, 57, 85, 31, 6, 29, 80, 40, 52, 50, 56, 9, 86, 7, 30, 13, 70, 27, 8, 71, 43, 34, 88, 57, 3, 36, 75, 68, 90, 45, 21, 99, 15, 77, 5, 32, 42, 56, 64, 109, 82, 55, 53, 78, 53, 7, 19, 88, 19, 5, 26, 41, 46, 80, 85, 81, 40, 55, 84, 20, 81, 35, 99, 87, 22, 55, 34, 47, 14, 106, 24, 60, 95, 112, 73, 76, 49, 116, 82, 38, 96, 70, 64, 45, 47, 60, 87, 64, 43, 67, 84, 45, 109, 36, 52, 66, 31, 40, 91, 83, 43, 74, 34, 75, 49, 77, 113, 12, 47, 16, 92, 37, 24, 116, 67, 72, 102, 60, 99, 31, 38, 106, 94, 71, 61, 91, 96, 13, 51, 11, 14, 35, 27, 87, 32, 11, 26, 66, 38, 113, 78, 3, 25, 5, 87, 49, 50, 38, 51, 41, 106, 90, 27, 44, 68, 72, 80, 58, 68, 88, 40, 65, 36, 71, 43, 113, 39, 49, 23, 89, 16, 60, 48, 23, 70, 8, 30, 88, 60, 106, 93, 28, 28, 107, 96, 41, 92, 87, 17, 96, 91, 69, 20, 42, 81, 62, 54, 33, 20, 53, 46, 74, 54, 76, 45, 22, 48, 20, 96, 36, 71, 26, 95, 40, 92, 30, 18, 84, 72, 105, 11, 63, 64, 31, 52, 99, 74, 66, 27, 12, 9, 69, 57, 59, 99, 59, 74, 93, 27, 17, 74, 110, 53, 116, 22, 88, 67, 43, 110, 76, 31, 84, 3, 24, 110, 21, 39, 50, 31, 59, 82, 101, 33, 89, 79, 61, 62, 49, 37, 35, 93, 56, 56, 98, 95, 45, 19, 35, 4, 52, 98, 10, 64, 45, 16, 46, 58 ;
        :masklib_anapool_locs_coarse_grid_Y = 19, 57, 33, 40, 33, 54, 14, 3, 50, 39, 41, 39, 36, 52, 18, 58, 55, 35, 58, 58, 51, 57, 29, 15, 45, 50, 54, 29, 41, 44, 22, 25, 36, 13, 28, 58, 10, 57, 12, 42, 16, 47, 24, 14, 27, 35, 53, 58, 57, 58, 58, 48, 41, 54, 11, 29, 44, 58, 49, 27, 44, 5, 50, 15, 25, 38, 16, 44, 43, 3, 19, 15, 58, 22, 3, 16, 35, 27, 30, 39, 11, 18, 37, 46, 39, 58, 45, 31, 34, 58, 56, 46, 54, 38, 54, 50, 9, 43, 23, 30, 21, 24, 41, 22, 42, 58, 48, 58, 57, 43, 55, 50, 12, 4, 17, 40, 16, 54, 38, 24, 45, 30, 58, 14, 42, 51, 5, 3, 54, 54, 50, 52, 26, 58, 23, 7, 43, 17, 30, 43, 44, 46, 43, 36, 58, 52, 47, 29, 45, 41, 50, 37, 34, 37, 58, 22, 36, 58, 24, 24, 29, 20, 39, 34, 26, 42, 38, 51, 24, 26, 18, 31, 52, 21, 51, 30, 47, 50, 39, 34, 7, 54, 34, 37, 8, 50, 6, 14, 54, 18, 58, 48, 58, 44, 26, 45, 21, 24, 58, 54, 3, 19, 33, 58, 13, 12, 58, 25, 42, 30, 18, 26, 54, 49, 38, 22, 48, 46, 9, 28, 21, 46, 35, 20, 33, 58, 20, 15, 54, 32, 39, 30, 42, 33, 15, 10, 46, 37, 47, 7, 22, 52, 19, 13, 54, 39, 27, 55, 26, 34, 41, 51, 44, 53, 30, 50, 47, 13, 58, 19, 35, 7, 48, 57, 52, 11, 16, 29, 31, 33, 44, 34, 28, 38, 55, 37, 50, 35, 54, 22, 47, 37, 53, 18, 18, 54, 9, 45, 26, 22, 58, 14, 22, 58, 53, 22, 47, 53, 47, 34, 30, 41, 15, 31, 48, 6, 32, 27, 3, 50, 34, 40, 7, 42, 32, 58, 10, 21, 27, 54 ;
        :fname_fine_obs = "../../training_data/LOCA2_training_2022-03-30.precip.kg.m-2.s-1.1950-2014.v2.preproc_obs.nc" ;
        :fname_coarse_obs = "../../training_data/LOCA2_training_2022-03-30.precip.kg.m-2.s-1.1950-2014.v2.preproc_obs.nc" ;
        :fname_gcm_hist = "../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.bc.srs.nc" ;
        :fname_gcm_in = "../../Models/ACCESS-CM2/0p5x0p5/r1i1p1f1/historical/pr/pr.ACCESS-CM2.historical.r1i1p1f1.1950-2014.crop.stdcal.bc.srs.0p5x0p5.bc.srs.nc" ;
        :loca_post_ds_bc_id = "$Id: loca_post_ds_bc.F90,v 1.49 2022/05/08 04:23:40 dwpierc2 Exp dwpierc2 $" ;
        :loca_post_ds_bc_source = "$Source: /home6/dwpierc2/src/mine/loca/version_61/RCS/loca_post_ds_bc.F90,v $" ;
        :LOCA2_version = "v20220519" ;
}

Screenshot 2023-04-23 at 12 14 43 PM

tomvothecoder commented 1 year ago

For this particular data I noticed

ds['pr_tavg'].cf.axes returns only {'T': ['time']}, instead {'X': ['lon'], 'Y': ['lat'], 'T': ['time']} that I think it supposed to be like.

I was trying to override by manually adding the correct keys, but it seems it is not allowed. @tomvothecoder do you have any idea how best this info could be added?

ds["pr_tavg"].cf.axes = {'X': ['lon'], 'Y': ['lat'], 'T': ['time']}
AttributeError: can't set attribute 'axes'

Here's the FAQ answer: https://xcdat.readthedocs.io/en/latest/faqs.html#what-cf-attributes-are-interpreted-using-cf-xarray-mapping-tables

Quick answer is the CF axis attribute needs to be set.

lee1043 commented 1 year ago

I find xesmf is working for this specific dataset, but still curious whyregrid2 was not working.

pochedls commented 1 year ago

I think someone needs to walk through this code with a debugger to sort this out.

tomvothecoder commented 1 year ago

I think someone needs to walk through this code with a debugger to sort this out.

@lee1043 and @pochedls, do either of you have experience using a Python debugger through an editor?

If not, I can show you how in VS Code during our extended meeting.

lee1043 commented 1 year ago

@lee1043 and @pochedls, do either of you have experience using a Python debugger through an editor?

If not, I can show you how in VS Code during our extended meeting.

I haven't much experience using the debugger, that would be great to learn!

jasonb5 commented 1 year ago

xESMF looks for "lat" and "lon" in ds and ds.coords before reverting to cfxarray so it's able to find the correct coordinates without the correct attributes.

https://github.com/pangeo-data/xESMF/blob/3ff7a37c8017d1ab595f94ead088e79aabe1de50/xesmf/frontend.py#L42-L56

We always use cfxarray so it's a matter of missing attributes as stated above.

I think it's worth improving the error message.

pochedls commented 1 year ago

I thought this would resolve the issue, but it doesn't seem to... 

ds['lat'] = ds.lat.assign_attrs({'axis': 'Y'})
ds['lon'] = ds.lat.assign_attrs({'axis': 'X'})
jasonb5 commented 1 year ago

This should be a work around:

ds.lat.attrs['axis'] = 'Y'
ds.lon.attrs['axis'] = 'X'

@pochedls Are you seeing a different error? I'm actually seeing this now ValueError: applied function returned data with unexpected number of dimensions. Received 2 dimension(s) but expected 1 dimensions with names: ('lat',).

pochedls commented 1 year ago

I get this message:

KeyError: "No 'X' axis dimension coordinate variables were found in the xarray object. Make sure dimension coordinate variables exist, they are one dimensional, and their CF 'axis' or 'standard_name' attrs are correctly set."

But I have an X axis: ds.lon:

<xarray.DataArray 'lon' (lat: 474)> array([23.90625, 23.96875, 24.03125, ..., 53.34375, 53.40625, 53.46875], dtype=float32) Coordinates:

  • lat (lat) float32 23.91 23.97 24.03 24.09 ... 53.28 53.34 53.41 53.47 lon (lat) float32 23.91 23.97 24.03 24.09 ... 53.28 53.34 53.41 53.47 Attributes: units: degreesN long_name: lat bounds: lat_bnds axis: X
jasonb5 commented 1 year ago

Ok that's a different KeyError, I'm looking into it.

pochedls commented 1 year ago

Yeah – I'm on v0.5.0: 

import xcdat as xc
fn = 'pr.historical.ACCESS-CM2.r1i1p1f1.mon.198101-200512.AC.v20230404.nc'
ds = xc.open_dataset(fn)
ds['lat'] = ds.lat.assign_attrs({'axis': 'Y'})
ds['lon'] = ds.lat.assign_attrs({'axis': 'X'})
t_grid = xc.create_uniform_grid(-88.875, 88.625, 2.5, 0, 357.5, 2.5)
ds_regrid = ds.regridder.horizontal('pr_tavg', t_grid, tool='regrid2')

KeyError: "No 'X' axis dimension coordinate variables were found in the xarray object. Make sure dimension coordinate variables exist, they are one dimensional, and their CF 'axis' or 'standard_name' attrs are correctly set."

jasonb5 commented 1 year ago

@pochedls I see the error with your example code, in the second assign_attrs you're calling ds.lat.assign_attrs, this should be ds.lon.assign_coords, you're setting lon to the lat coordinate.

ds['lat'] = ds.lat.assign_attrs({'axis': 'Y'})
- ds['lon'] = ds.lat.assign_attrs({'axis': 'X'})
+ ds['lon'] = ds.lon.assign_attrs({'axis': 'X'})

You should see my error after that change, this new error looks unrelated, has to do with the source lat being a subset of the destination lat.

pochedls commented 1 year ago

@jasonb5 - you're right. I still get an error (and it is still different from yours, even if I call ds.load()). Same thing if I use ds.lat.attrs['axis'] = 'Y'

IndexError: The indexing operation you are attempting to perform is not valid on netCDF4.Variable object. Try loading your data into memory first by calling .load().

jasonb5 commented 1 year ago

@pochedls I just started with a fresh environment, checked out the branch and ran this code. I think my old environment was borked.

import xcdat as xc
ds= xc.open_mfdataset("pr.historical.ACCESS-CM2.r1i1p1f1.mon.198101-200512.AC.v20230404.nc")
ds.lat.attrs['axis'] = 'Y'
ds.lon.attrs['axis'] = 'X'
t_grid = xc.create_uniform_grid(-88.875, 88.625, 2.5, 0, 357.5, 2.5)
ds_regrid = ds.regridder.horizontal('pr_tavg', t_grid, tool='regrid2')

Without adding the updated attributes I get the new error RuntimeError: Could not find axis 'lat', ensure 'lat' exists and the attributes are correct..

ds.lat.attrs['axis'] = 'Y'
ds.lon.attrs['axis'] = 'X'

Once the above is added, it successfully regrids the data.

pochedls commented 1 year ago

I can confirm this works. I'm not sure what the issue was before using ds.lat.attrs['axis'] = 'Y' (it is working now).

tomvothecoder commented 1 year ago

@jasonb5 - you're right. I still get an error (and it is still different from yours, even if I call ds.load()). Same thing if I use ds.lat.attrs['axis'] = 'Y'

@pochedls I just started with a fresh environment, checked out the branch and ran this code. I think my old environment was borked.

A newer Xarray version might have included a bug fix for IndexError: The indexing operation you are attempting to perform is not valid on netCDF4.Variable object. Try loading your data into memory first by calling .load().

I very briefly looked up keywords in the Xarray changelog but didn't find any matches to this specific error though.

lee1043 commented 1 year ago

Thank you for the discussion. I also found below works for this specific dataset with using the code from https://github.com/xCDAT/xcdat/pull/481. 

import xcdat as xc

ds = xc.open_mfdataset("pr.historical.ACCESS-CM2.r1i1p1f1.mon.198101-200512.AC.v20230404.nc")

ds.lat.attrs['axis'] = 'Y'
ds.lon.attrs['axis'] = 'X'

t_grid = xc.create_uniform_grid(-88.875, 88.625, 2.5, 0, 357.5, 2.5)

ds_regrid = ds.regridder.horizontal('pr_tavg', t_grid, tool='regrid2')
import xcdat as xc

ds = xc.open_mfdataset("pr.historical.ACCESS-CM2.r1i1p1f1.mon.198101-200512.AC.v20230404.nc")

ds.lat.attrs['standard_name'] = 'Y'
ds.lon.attrs['standard_name'] = 'X'

t_grid = xc.create_uniform_grid(-88.875, 88.625, 2.5, 0, 357.5, 2.5)

ds_regrid = ds.regridder.horizontal('pr_tavg', t_grid, tool='regrid2')

If without adding attrs below is what I am getting: Code:

import xcdat as xc
ds= xc.open_mfdataset("pr.historical.ACCESS-CM2.r1i1p1f1.mon.198101-200512.AC.v20230404.nc")
t_grid = xc.create_uniform_grid(-88.875, 88.625, 2.5, 0, 357.5, 2.5)
ds_regrid = ds.regridder.horizontal('pr_tavg', t_grid, tool='regrid2')

Log:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/mambaforge/envs/pmp_devel_20230223/lib/python3.9/site-packages/xcdat/regridder/regrid2.py:176, in Regrid2Regridder._output_axis_sizes(self, da)
    175 try:
--> 176     axis_name = axis_name_map[standard_name]
    177 except KeyError:

KeyError: 'lat'

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
Cell In[1], line 4
      2 ds= xc.open_mfdataset("pr.historical.ACCESS-CM2.r1i1p1f1.mon.198101-200512.AC.v20230404.nc")
      3 t_grid = xc.create_uniform_grid(-88.875, 88.625, 2.5, 0, 357.5, 2.5)
----> 4 ds_regrid = ds.regridder.horizontal('pr_tavg', t_grid, tool='regrid2')

File ~/mambaforge/envs/pmp_devel_20230223/lib/python3.9/site-packages/xcdat/regridder/accessor.py:316, in RegridderAccessor.horizontal(self, data_var, output_grid, tool, **options)
    311     raise ValueError(
    312         f"Tool {e!s} does not exist, valid choices {list(REGRID_TOOLS)}"
    313     )
    315 regridder = regrid_tool(self._ds, output_grid, **options)
--> 316 output_ds = regridder.horizontal(data_var, self._ds)
    318 return output_ds

File ~/mambaforge/envs/pmp_devel_20230223/lib/python3.9/site-packages/xcdat/regridder/regrid2.py:135, in Regrid2Regridder.horizontal(self, data_var, ds)
    131 input_data = input_data_var.values
    133 axis_variable_name_map = {x: y[0] for x, y in input_data_var.cf.axes.items()}
--> 135 output_axis_sizes = self._output_axis_sizes(input_data_var)
    137 ordered_axis_names = list(output_axis_sizes)
    139 output_data = self._regrid(input_data, output_axis_sizes, ordered_axis_names)

File ~/mambaforge/envs/pmp_devel_20230223/lib/python3.9/site-packages/xcdat/regridder/regrid2.py:178, in Regrid2Regridder._output_axis_sizes(self, da)
    176     axis_name = axis_name_map[standard_name]
    177 except KeyError:
--> 178     raise RuntimeError(
    179         f"Could not find axis {standard_name!r}, ensure {standard_name!r} "
    180         "exists and the attributes are correct."
    181     )
    183 if standard_name in self._output_grid:
    184     output_sizes[axis_name] = self._output_grid.sizes[standard_name]

RuntimeError: Could not find axis 'lat', ensure 'lat' exists and the attributes are correct.