ESMValGroup / ESMValCore

ESMValCore: A community tool for pre-processing data from Earth system models in CMIP and running analysis scripts.
https://www.esmvaltool.org
Apache License 2.0
42 stars 38 forks source link

Anomalies preprocessor with period 'full' removes important metadata from cube #651

Open Peter9192 opened 4 years ago

Peter9192 commented 4 years ago

Describe the bug When the anomalies preprocessor function is applied with period=full, it removes important metadata such as the variable name, all attributes, etc. This doesn't happen for other periods like month or day. Consequently, subsequent preprocessors or diagnostics fail.

I think it's happening here:

cube = cube - reference

and this seems to be the intended behaviour in Iris, as it is documented here.

Perhaps EMSValTool should only subtract the data instead of the full cube?

Example print(cube):

air_temperature / (K)               (time: 1668)
     Dimension coordinates:
          time                           x
     Auxiliary coordinates:
          day_of_year                    x
          month_number                   x
     Scalar coordinates:
          height: 1.5 m
          latitude: 0.0 degrees, bound=(-90.0, 90.0) degrees
          longitude: 179.0625 degrees, bound=(-0.9375, 359.0625) degrees
     Attributes:
          Conventions: CF-1.7
          associated_files: baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_ACCESS1-0_rcp45_r0i0p0.nc...
          branch_time: 732311.0;109207.0;732311.0
          cmor_version: 2.8.0
          contact: The ACCESS wiki: http://wiki.csiro.au/confluence/display/ACCESS/Home. Contact...
          experiment: RCP4.5;historical;RCP4.5
          experiment_id: rcp45;historical;rcp45
          forcing: GHG, Oz, SA, Sl, Vl, BC, OC, (GHG = CO2, N2O, CH4, CFC11, CFC12, CFC113,...
          frequency: mon
          initialization_method: 1
          institute_id: CSIRO-BOM
          institution: CSIRO (Commonwealth Scientific and Industrial Research Organisation, Australia),...
          model_id: ACCESS1-0
          modeling_realm: atmos
          parent_experiment: historical;pre-industrial control;historical
          parent_experiment_id: historical;piControl;historical
          parent_experiment_rip: r1i1p1
          physics_version: 1
          product: output
          project_id: CMIP5
          realization: 1
          references: See http://wiki.csiro.au/confluence/display/ACCESS/ACCESS+Publications
          source: ACCESS1-0 2011. Atmosphere: AGCM v1.0 (N96 grid-point, 1.875 degrees EW...
          table_id: Table Amon (27 April 2011) 9c851218e3842df9a62ef38b1e2575bb
          title: ACCESS1-0 model output prepared for CMIP5 RCP4.5;ACCESS1-0 model output...
          version_number: v20120115
     Cell methods:
          mean: time
          mean: longitude, latitude
anoms = anomalies(cube, period='full', reference=dict(start_year=1980, start_month=1, start_day=1, end_year=2009, end_month=12, end_day=31))
print(anoms)

gives

unknown / (K)                       (time: 1668)
     Dimension coordinates:
          time                           x
     Auxiliary coordinates:
          month_number                   x
     Scalar coordinates:
          height: 1.5 m
          latitude: 0.0 degrees, bound=(-90.0, 90.0) degrees
          longitude: 179.0625 degrees, bound=(-0.9375, 359.0625) degrees
valeriupredoi commented 4 years ago

ugh cheers @Peter9192 - definitely a bug! Let me create a fix :beer:

Peter9192 commented 3 years ago

Reopening this one as I just tried to use a similar recipe with standardize=True, which results in

variables:
        double unknown ;
                unknown:_FillValue = 1.e+20 ;
                unknown:units = "1" ;
                unknown:cell_methods = "time: mean" ;
                unknown:coordinates = "lat lon time" ;

Not sure if this was introduced by a change in Iris, or that it never really worked but we didn't catch it. We also apparently don't cover this case in the tests yet.

So the core issue being that the variable name changed to 'unknown'