search

geoschem / GCHP

The "superproject" wrapper repository for GCHP, the high-performance instance of the GEOS-Chem chemical-transport model.
https://gchp.readthedocs.io
Other
23 stars 26 forks source link

[BUG/ISSUE] Changes to HISTORY.rc not reflected in output netcdf files #130

Closed cbutenhoff closed 3 years ago

cbutenhoff commented 3 years ago

I'm clearly not setting some configuration correctly because the changes I make in my HISTORY.rc are not being reflected in the output netcdf files. In particular, when setting the level and fields in both the Emissions and SpecConc collections like this:

# Emissions (see HEMCO_Diagn.rc for additional config settings)
  Emissions.template:       '%y4%m2%d2_%h2%n2z.nc4',
  Emissions.format:         'CFIO',
  Emissions.timestampStart: .true.
  Emissions.monthly:        0
  Emissions.frequency:      010000
  Emissions.duration:       010000
  Emissions.mode:           'time-averaged'
 # Emissions.grid_label:     'GLOBAL4x5'
  Emissions.levels:  1
  Emissions.fields:       'EmisC2H6_Total       ', 'GCHPchem',
                          'EmisC2H6_Aircraft    ', 'GCHPchem',
                          'EmisC2H6_Anthro      ', 'GCHPchem',
                          'EmisC2H6_BioBurn     ', 'GCHPchem',
                          'EmisC2H6_Ship        ', 'GCHPchem',

data are still written out at all levels and species:

[cbuten@login2 OutputDir]$ ncdump -h GEOSChem.Emissions.20190701_2300z.nc4 | more
netcdf GEOSChem.Emissions.20190701_2300z {
dimensions:
        XCdim = 49 ;
        Xdim = 48 ;
        YCdim = 49 ;
        Ydim = 48 ;
        lev = 72 ;
        ncontact = 4 ;
        nf = 6 ;
        orientationStrLen = 5 ;
        time = UNLIMITED ; // (1 currently)
...
float EmisACET_Anthro(time, lev, nf, Ydim, Xdim) ;
                EmisACET_Anthro:_FillValue = 1.e+15f ;
                EmisACET_Anthro:coordinates = "lons lats" ;
                EmisACET_Anthro:grid_mapping = "cubed_sphere" ;
                EmisACET_Anthro:long_name = "ACET emission flux from anthropogenic" ;
                EmisACET_Anthro:missing_value = 1.e+15f ;
                EmisACET_Anthro:units = "kg/m2/s" ;
                EmisACET_Anthro:valid_range = -1.e+15f, 1.e+15f ;
        float EmisACET_BioBurn(time, nf, Ydim, Xdim) ;
                EmisACET_BioBurn:_FillValue = 1.e+15f ;
                EmisACET_BioBurn:coordinates = "lons lats" ;
                EmisACET_BioBurn:grid_mapping = "cubed_sphere" ;
                EmisACET_BioBurn:long_name = "ACET emission flux from biomass burning" ;
                EmisACET_BioBurn:missing_value = 1.e+15f ;
                EmisACET_BioBurn:units = "kg/m2/s" ;
                EmisACET_BioBurn:valid_range = -1.e+15f, 1.e+15f ;
        float EmisACET_Biogenic(time, nf, Ydim, Xdim) ;
                EmisACET_Biogenic:_FillValue = 1.e+15f ;
                EmisACET_Biogenic:coordinates = "lons lats" ;
                EmisACET_Biogenic:grid_mapping = "cubed_sphere" ;
                EmisACET_Biogenic:long_name = "ACET emission flux from biogenic sources" ;
                EmisACET_Biogenic:missing_value = 1.e+15f ;
                EmisACET_Biogenic:units = "kg/m2/s" ;
                EmisACET_Biogenic:valid_range = -1.e+15f, 1.e+15f ;

It's also odd that XDIM and YDIM are 48 even though I am running C24 resolutions. Also the global attribute gridspec_file = "C48_gridspec.nc4", when I would expect "C24_gridspec.nc4".

The log file indicates the changes I made to HISTORY.rc are being read in 

Reading HISTORY RC Files:
 -------------------------
 NOT using buffer I/O for file: HISTORY.rc
 NOT using buffer I/O for file: Emissions.rcx
 NOT using buffer I/O for file: Metrics.rcx
 NOT using buffer I/O for file: SpeciesConc.rcx

 Freq: 00010000  Dur: 00010000  TM:   -1  Collection: Emissions
 Freq: 00010000  Dur: 00010000  TM:   -1  Collection: Metrics
 Freq: 00010000  Dur: 00010000  TM:   -1  Collection: SpeciesConc

 Independent Output Export States:
 ---------------------------------
           1 GCHPchem

 Initializing Output Stream: Emissions
 ---------------------------
       Format: CFIO
         Mode: time-averaged
        Nbits:          100
       Slices:            5
      Deflate:            0
    Frequency:        10000
 Acc_Interval:        10000
     Ref_Date:     20190701
     Ref_Time:            0
     Duration:        10000
    XY-offset:            0   (DcPc: Dateline Center, Pole Center)
      Fields: EmisC2H6_Total EmisC2H6_Aircraft EmisC2H6_Anthro EmisC2H6_BioBurn EmisC2H6_Ship
    Vertical  Levels:            1

and the contents of Emissions.rcx in my run directory also indicate this:

[cbuten@login2 gchp_merra2_fullchem]$ more Emissions.rcx
  Emissions.template:       '%y4%m2%d2_%h2%n2z.nc4',
  Emissions.format:         'CFIO',
  Emissions.timestampStart: .true.
  Emissions.monthly:        0
  Emissions.frequency:      010000
  Emissions.duration:       010000
  Emissions.mode:           'time-averaged'
  Emissions.levels:  1
  Emissions.fields:       'EmisC2H6_Total       ', 'GCHPchem',
                          'EmisC2H6_Aircraft    ', 'GCHPchem',
                          'EmisC2H6_Anthro      ', 'GCHPchem',
                          'EmisC2H6_BioBurn     ', 'GCHPchem',
                          'EmisC2H6_Ship        ', 'GCHPchem',

An additional unrelated issue with HISTORY.rc is when I set the grid_label in HISTORY.rc:

#  lat-lon global grid at 4x5 resolution
    GLOBAL4x5.GRID_TYPE: LatLon
    GLOBAL4x5.IM_WORLD: 72 
    GLOBAL4x5.JM_WORLD: 46
    GLOBAL4x5.POLE: PC
    GLOBAL4x5.DATELINE: DC
    GLOBAL4x5.LM: 72
...

# Emissions (see HEMCO_Diagn.rc for additional config settings)
  Emissions.template:       '%y4%m2%d2_%h2%n2z.nc4',
  Emissions.format:         'CFIO',
  Emissions.timestampStart: .true.
  Emissions.monthly:        0
  Emissions.frequency:      010000
  Emissions.duration:       010000
  Emissions.mode:           'time-averaged'
  Emissions.grid_label:     GLOBAL4x5
  Emissions.levels:  1
  Emissions.fields:       'EmisC2H6_Total       ', 'GCHPchem',
                          'EmisC2H6_Aircraft    ', 'GCHPchem',
                          'EmisC2H6_Anthro      ', 'GCHPchem',
                          'EmisC2H6_BioBurn     ', 'GCHPchem',
                          'EmisC2H6_Ship        ', 'GCHPchem',

the job fails with these errors, whether I specify the grid_label name in quotes or not:

Reading HISTORY RC Files:
 -------------------------
 NOT using buffer I/O for file: HISTORY.rc
 NOT using buffer I/O for file: Emissions.rcx
 NOT using buffer I/O for file: Metrics.rcx
 NOT using buffer I/O for file: SpeciesConc.rcx
pe=00011 FAIL at line=01077    MAPL_HistoryGridComp.F90                 <needs informative message>
pe=00003 FAIL at line=01077    MAPL_HistoryGridComp.F90                 <needs informative message>
pe=00015 FAIL at line=01077    MAPL_HistoryGridComp.F90                 <needs informative message>
pe=00003 FAIL at line=01839    MAPL_Generic.F90                         <needs informative message>
pe=00003 FAIL at line=00673    MAPL_CapGridComp.F90                     <status=1>
pe=00015 FAIL at line=01839    MAPL_Generic.F90                         <needs informative message>
pe=00003 FAIL at line=00618    MAPL_CapGridComp.F90                     <status=1>
pe=00015 FAIL at line=00673    MAPL_CapGridComp.F90                     <status=1>
pe=00015 FAIL at line=00618    MAPL_CapGridComp.F90                     <status=1>
pe=00015 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
pe=00015 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
pe=00015 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
pe=00003 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
pe=00003 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
pe=00003 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
pe=00003 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
pe=00003 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
pe=00003 FAIL at line=00030    GCHPctm.F90                              <status=1>
pe=00005 FAIL at line=01077    MAPL_HistoryGridComp.F90                 <needs informativ

GCHP v13.1.2 HISTORY.rc.txt runlog: gchp_5456132.out.txt runConfig.sh.txt

LiamBindle commented 3 years ago

@cbutenhoff Recently I've been playing around with configuring HISTORY.rc, so I sympathize with it being tricky.

A couple things I see right away:

  1. The GRID_LABELS: declaration on line 20 is missing your GLOBAL4x5 tag. I think this is the source of your crash.
  2. In your GRID4x5 definition you include LM:72. I think this might be what's causing the 72 level output. I would recommend omitting LM and treating the grid labels as horizontal grid definitions only.
  3. I don't put my grid labels in single quotes. It might be okay with single quotes, but I would try without.

When I'm writing a custom HISTORY.rc I find the default file a bit overwhelming. I'd recommend stripping out everything you don't need so it's easier to keep track of. Here's a start

EXPID:  OutputDir/GEOSChem
EXPDSC: GEOS-Chem_devel
CoresPerNode: 18
VERSION: 1

GRID_LABELS: GLOBAL4x5
    ::
    GLOBAL4x5.GRID_TYPE:  LatLon
    GLOBAL4x5.IM_WORLD:   72 
    GLOBAL4x5.JM_WORLD:   46
    GLOBAL4x5.POLE:       PC
    GLOBAL4x5.DATELINE:   DC

COLLECTIONS: 'MyEmissionsCollection',
             'MySpeciesCollection',
::
# Note: 'Emis*' variables are upside-down relative to everything else (therefore, pick level 72 rather than 1)
  MyEmissionsCollection.template:     '%y4%m2%d2_%h2%n2z.nc4',
  MyEmissionsCollection.format:       'CFIO',
  MyEmissionsCollection.frequency:    010000
  MyEmissionsCollection.duration:     240000
  MyEmissionsCollection.mode:         'time-averaged'
  MyEmissionsCollection.grid_label:   GLOBAL4x5
  MyEmissionsCollection.conservative:  1
  MyEmissionsCollection.levels:       72
  MyEmissionsCollection.fields:       'EmisC2H6_Total       ', 'GCHPchem',
                                      'EmisC2H6_Aircraft    ', 'GCHPchem',
                                      'EmisC2H6_Anthro      ', 'GCHPchem',
                                      'EmisC2H6_BioBurn     ', 'GCHPchem',
                                      'EmisC2H6_Ship        ', 'GCHPchem',                        
::
  MySpeciesCollection.template:       '%y4%m2%d2_%h2%n2z.nc4',
  MySpeciesCollection.format:         'CFIO',
  MySpeciesCollection.frequency:      010000
  MySpeciesCollection.duration:       240000
  MySpeciesCollection.mode:           'time-averaged'
  MySpeciesCollection.grid_label:     GLOBAL4x5
  MySpeciesCollection.conservative:    1
  MySpeciesCollection.levels:          1
  MySpeciesCollection.fields:       'SpeciesConc_C2H6          ', 'GCHPchem',                    
::

Note Here I put 010000 for the frequency (hourly output) and 240000 for the duration. This means you'll get 1 file per day with 24 time slices.

Give that a try. Hopefully I don't have any syntax errors in there.

Also, I update this page: https://gchp.readthedocs.io/en/latest/supplement/config-files/HISTORY.html on Friday. If you haven't read the updated version of it, it might be a useful reference.

Let me know how that goes.

lizziel commented 3 years ago

Hi @cbutenhoff, regarding the issue of your output file grid size not matching your run grid size, that is indeed bizarre. The log file consistently indicates C24 throughout. I did a test with a dev version of 13.2, but that uses the same MAPL as 13.1.2, where I simply added levels: 1 to a couple collections (Emissions and SpeciesConc) and ran at C24. The output file had Xdim and Ydim values of 24.

Could you try a rerun with a clean run directory to see if you still see this behavior? I'm wondering if maybe you had some old C48 run files hanging around.

cbutenhoff commented 3 years ago

Thanks a lot for your help. I used Liam's file and that now works.

You might be right about having old C48 files lying around since I used the same directory for the C24 runs. Do you still want me to run using a clean directory to further diagnose the problem?

lizziel commented 3 years ago

No need to further diagnose the grid size problem if you think looking at an old file was a possibility. Just let us know if you see the same thing again. I'll close out this issue now.