search

geoschem / GCHP

The "superproject" wrapper repository for GCHP, the high-performance instance of the GEOS-Chem chemical-transport model.
https://gchp.readthedocs.io
Other
22 stars 25 forks source link

[BUG/ISSUE] Failure at EXTDATA Initialization Stage When Utilizing External Emission Inventories #166

Closed JbourJabber closed 2 years ago

JbourJabber commented 2 years ago

I am working on conducting an annual simulation of global atmospheric C2H6 and I am attempting to import additional emission inventories into HEMCO--which I believe is causing my runs to crash prematurely. Currently I have successfully run a baseline emission scenario without any additional emission inventories, but as soon as I incorporate them into the HEMCO_Config.rc and ExtData.rc--the runs are rendered non-functional.

Following the steps detailed in Ex. 1 from the HEMCO User Guide, (http://wiki.seas.harvard.edu/geos-chem/index.php/The_HEMCO_User%27s_Guide#Overview), here are the following edited sections:

HEMCO_Config.rc:

> ###############################################################################
> ### BEGIN SECTION EXTENSION SWITCHES
> ###############################################################################
> ExtNr ExtName                on/off  Species  Years avail.
> 0       Base                   : on    *
>     --> MEGAN-MACC_Biogenic_C2H6 :     true  C2H6  # 1980-2000
>     --> MEGANv2_Biogenic_C2H6  :       true  C2H6  # 2000
>     --> POET_Oceanic_C2H6      :       false  C2H6  # 1990
>     --> RETRO_Oceanic_C2H6     :       true  C2H6  # 1960-2000
>     --> Etiope_Geologic_C2H6   :       true  C2H6  # 2018
>
> ###############################################################################
> ### BEGIN SECTION BASE EMISSIONS
> ###############################################################################
> 
> ExtNr Name sourceFile sourceVar sourceTime C/R/E SrcDim SrcUnit Species ScalIDs Cat Hier
> 
> (((EMISSIONS
> 
> (((MEGAN-MACC_Biogenic_C2H6
> 0 MEGAN-MACC_C2H6_emiss_bio  $ROOT/MEGAN-MACC_Biogenic_C2H6/MEGAN-MACC_Glb_0.5x0.5_bio_ethane__monthly.nc  emiss_bio  1980-2020/1-12/1-31/0-23  C  xy  kg/m2/s  C2H6  1 1 1
> )))MEGAN-MACC_Biogenic_C2H6
> 
> (((MEGANv2_Biogenic_C2H6
> 0 MEGANv2_C2H6_emiss_bio  $ROOT/MEGANv2_Biogenic_C2H6/MEGANv2_Glb_0.5x0.5_bio_ethane__monthly.nc  emiss_bio  2000/1-12/1-31/0-23  C  xy  kg/m2/s  C2H6  1 1 1
> )))MEGANv2_Biogenic_C2H6
> 
> (((POET_Oceanic_C2H6
> 0 POET_C2H6_emiss_ocean  $ROOT/POET_Oceanic_C2H6/POET_Glb_1x1_oce_ethane__yearly.nc  emiss_ocean  1990/1-12/1-31/0-23  C  xy  kg/m2/s  C2H6  1 1 1
> )))POET_Oceanic_C2H6
> 
> (((RETRO_Oceanic_C2H6
> 0 RETRO_C2H6_emiss_ocean  $ROOT/RETRO_Oceanic_C2H6/RETRO_Glb_0.5x0.5_oce_ETHANE__monthly.nc  ocean  1960-2000/1-12/1-31/0-23  C  xy  kg/m2/s  C2H6  1 1 1
> )))RETRO_Oceanic_C2H6
> 
> (((Etiope_Geologic_C2H6
> 0 Etiope_C2H6_emiss_geo  $ROOT/Etiope_Geologic_C2H6/Total_GEOC2H6_Output_2018.nc  C2H6_Output  2018/1-12/1-31/0-23  C  xy  kg/m2/s  C2H6  - 1 1
> )))Etiope_Geologic_C2H6
>
> ###############################################################################
> ### BEGIN SECTION SCALE FACTORS
> ###############################################################################
> 
> ScalID Name sourceFile sourceVar sourceTime C/R/E SrcDim SrcUnit Oper
> 
> (((EMISSIONS
> 
> %%% Hourly factors, read from disk %%%
> 1 HOURLY_SCALFACT $ROOT/hourly.nc factor 2000/1/1/0-23    C xy 1 1    
> 
> #=============================================================================

ExtData.rc:

> ###############################################################################
> ###
> ### HEMCO Base Emissions (Update if HEMCO_Config.rc changes)
> ###
> ###############################################################################
> #
> #
> #==============================================================================
>  --- MEGAN-MACC_Biogenic_C2H6 (MEGAN-MACC) ---
> #==============================================================================
> MEGAN-MACC_BIO_C2H6  kg/m2/s  Y  Y none none emiss_bio /scratch/ExtData/HEMCO/MEGAN-MACC_Biogenic_C2H6/MEGAN-MACC_Glb_0.5x0.5_bio_ethane__monthly.nc
> #
> #==============================================================================
>  --- MEGANv2_Biogenic_C2H6 (MEGANv2) ---
> #==============================================================================
> MEGANv2_BIO_C2H6  kg/m2/s  Y  Y none none emiss_bio /scratch/ExtData/ExtData/HEMCO/MEGANv2_Biogenic_C2H6/MEGANv2_Glb_0.5x0.5_bio_ethane__monthly.nc
> #
> #==============================================================================
>  --- POET_Oceanic_C2H6 (POET) ---
> #==============================================================================
> POET_OCE_C2H6  kg/m2/s  Y  Y none none emiss_ocean /scratch/ExtData/HEMCO/POET_Oceanic_C2H6/POET_Glb_1x1_oce_ethane__yearly.nc
> #
> #==============================================================================
>  --- RETRO_Oceanic_C2H6 (RETRO) ---
> #==============================================================================
> RETRO_OCE_C2H6  kg/m2/s  Y  Y none none ocean /scratch/ExtData/HEMCO/RETRO_Oceanic_C2H6/RETRO_Glb_0.5x0.5_oce_ETHANE__monthly.nc
> #
> #==============================================================================
>  --- Etiope_Geologic_C2H6 (ETIOPE) ---
> #==============================================================================
> Etiope_GEO_C2H6  kg/m2/s  Y  Y none none C2H6_Output /scratch/ExtData/HEMCO/Etiope_Geologic_C2H6/Total_GEOC2H6_Output_2018.nc
> #
> #=============================================================================

As I then proceed to submit my slurm-job, the following output is produced upon failure of the job.

>
 GENERIC: INFO: Started the  'Initialize' stage of the gridded component 'EXTDATA'
>                                                           Mem/Swap Used (MB) at EXTDATAMAPL_GenericInitialize=  1.095E+05  6.000E+00
> pe=00037 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00051 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00051 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00051 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00051 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00051 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00051 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00051 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00051 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00051 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00051 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00037 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00037 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00037 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00037 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00037 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00037 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00037 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00037 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00037 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00001 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00001 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00001 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00001 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00005 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00011 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00011 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00011 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00011 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00011 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00011 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00001 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00001 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00001 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00001 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00001 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00001 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00009 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00009 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00009 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00009 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00011 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00011 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00011 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00011 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00009 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00009 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00009 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00009 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00009 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00009 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00005 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00005 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00005 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00005 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00005 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00005 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00005 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00005 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00005 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00013 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00013 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00013 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00013 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00049 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00049 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00049 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00049 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00013 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00013 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00013 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00013 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00013 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00013 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00049 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00049 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00049 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00049 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00049 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00049 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00043 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00043 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00043 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00043 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00043 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00043 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00043 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00043 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00043 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00043 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00012 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00012 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00012 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00012 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00012 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00012 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00012 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00012 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00012 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00012 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00019 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00000 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00000 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00014 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00000 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00000 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00014 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00000 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00000 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00014 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00014 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00014 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00014 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00000 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00000 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00000 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00000 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00008 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00008 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00014 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00014 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00044 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00044 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00044 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00044 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00044 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00008 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00008 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00008 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00008 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00014 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00014 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00003 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00003 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00003 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00008 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00008 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00008 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00008 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00052 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00003 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00029 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00029 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00029 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00029 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00029 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00029 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00029 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00029 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00029 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00029 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00003 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00003 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00003 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00003 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00003 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00027 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00027 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00027 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00027 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00027 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00027 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00027 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00027 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00027 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00027 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00003 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00034 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00034 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00034 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00034 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00034 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00034 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00034 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00034 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00046 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00046 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00046 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00046 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00046 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00046 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00046 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00046 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00046 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00044 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00044 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00044 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00044 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00044 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00034 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00034 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00035 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00035 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00035 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00035 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00047 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00047 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00047 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00028 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00028 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00028 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00046 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00035 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00035 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00035 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00035 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00035 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00035 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00039 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00039 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00039 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00028 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00028 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00028 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00028 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00047 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00047 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00047 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00047 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00025 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00025 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00025 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00039 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00039 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00039 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00039 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00028 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00028 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00028 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00047 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00047 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00047 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00002 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00002 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00025 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00025 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00025 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00045 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00002 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00002 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00025 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00025 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00025 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00025 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00039 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00039 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00039 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00002 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00002 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00002 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00002 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00002 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00019 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00019 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00019 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00045 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00045 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00045 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00002 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00019 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00019 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00019 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00019 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00019 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00019 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00045 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00045 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00045 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00030 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00030 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00030 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00030 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>

> --------------------------------------------------------------------------
> MPI_ABORT was invoked on rank 11 in communicator MPI_COMM_WORLD
> with errorcode 0.
> 
> NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
> You may or may not see output from other processes, depending on
> exactly when Open MPI kills them.
> --------------------------------------------------------------------------
> pe=00045 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00045 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00045 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00030 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00030 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00030 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00040 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00040 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00030 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00030 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00030 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00040 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00040 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00021 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00021 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00040 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00040 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00040 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00040 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00040 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00021 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00021 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00021 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00021 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00021 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00021 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00021 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00021 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00040 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00026 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00026 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00026 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00026 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00053 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00053 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00053 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00024 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00024 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00024 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00024 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00024 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00053 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00053 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00032 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00032 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00032 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00053 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00053 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00053 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00053 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00026 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00026 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00026 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00026 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00026 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00026 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00053 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00017 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00017 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00017 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00017 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00017 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00017 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00032 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00032 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00052 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00052 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00052 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00024 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00024 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00024 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00024 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00024 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00017 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00017 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00017 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00052 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00052 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00052 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00052 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00052 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00052 FAIL at line=00031    GCHPctm.F90                              <status=1>
> [compute097.cluster:21512] [[60789,0],0] ORTE_ERROR_LOG: Out of resource in file util/show_help.c at line 500
> pe=00032 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00032 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00032 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00032 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00032 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00017 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00020 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00023 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00020 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00020 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00020 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00018 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00018 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00018 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00023 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00023 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00023 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00020 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00020 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00020 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00020 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00020 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00023 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00023 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00023 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00018 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00018 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00018 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00018 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00020 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00023 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00023 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00023 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00018 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00018 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00018 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00033 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00033 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00033 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00033 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00033 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00033 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00033 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00033 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00033 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00036 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00033 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00042 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00042 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00042 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00031 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00031 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00042 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00042 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00042 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00042 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00031 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00031 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00031 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00031 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00031 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00031 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00031 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00031 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00004 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00004 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00042 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00042 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00042 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00004 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00004 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00004 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00004 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00036 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00004 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00004 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00004 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00038 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00038 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00038 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00038 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00004 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00038 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00038 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00038 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00038 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00036 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00036 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00038 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00038 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00036 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00036 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00036 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00036 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00036 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00036 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00050 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00050 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00050 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00050 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00050 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00050 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00050 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00050 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00050 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00050 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00041 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00041 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00041 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00041 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00041 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00041 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00041 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00041 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00041 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00041 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00048 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00048 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00048 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00048 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00048 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00048 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00048 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00048 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00048 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00048 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00006 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00006 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00006 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00006 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00006 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00006 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00006 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00006 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00006 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00006 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00007 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00007 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00007 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00007 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00007 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00007 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00007 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00007 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00007 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00007 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00010 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00010 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00010 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00010 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00010 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00010 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00010 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00010 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00010 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00010 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00015 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00015 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00015 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00015 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00015 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00015 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00015 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00015 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00015 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00015 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00016 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00016 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00016 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00016 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00016 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00016 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00016 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00016 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00016 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00016 FAIL at line=00031    GCHPctm.F90                              <status=1>
> pe=00022 FAIL at line=00688    MAPL_ExtDataGridCompMod.F90              <status=41>
> pe=00022 FAIL at line=01844    MAPL_Generic.F90                         <Error during the 'Initialize' stage of the gridded component 'EXTDATA'>
> pe=00022 FAIL at line=00766    MAPL_CapGridComp.F90                     <status=1>
> pe=00022 FAIL at line=00621    MAPL_CapGridComp.F90                     <status=1>
> pe=00022 FAIL at line=00908    MAPL_CapGridComp.F90                     <status=1>
> pe=00022 FAIL at line=00245    MAPL_Cap.F90                             <status=1>
> pe=00022 FAIL at line=00211    MAPL_Cap.F90                             <status=1>
> pe=00022 FAIL at line=00154    MAPL_Cap.F90                             <status=1>
> pe=00022 FAIL at line=00129    MAPL_Cap.F90                             <status=1>
> pe=00022 FAIL at line=00031    GCHPctm.F90                              <status=1>
> [compute097.cluster:21512] 52 more processes have sent help message help-mpi-api.txt / mpi-abort
> [compute097.cluster:21512] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages

Any assistance with this issue would be immensely appreciated, as I am not sure where I went wrong.

Thank you in advance!

lizziel commented 2 years ago

Hi @JbourJabber, thanks for reaching out about this. Please try the following to give me a bit more information.

  1. Set the Verbose and Warnings settings in HEMCO_Config.rc to 3 to send the maximum number of prints to output file HEMCO.log.
  2. Set CAP.EXTDATA option root_level in config file logging.yml to DEBUG to send root thread MAPL ExtData (input) prints to output file allPEs.log.
  3. Re-run the model
  4. Post the following files on this thread: HEMCO.log, allPEs.log, gchp.log, ExtData.rc, and HEMCO.rc. If you add extension .txt to the filenames you can drag and drop the file directly into the GitHub comment box.
JbourJabber commented 2 years ago

Hi @lizziel , thank you for following up this thread. I am looking forward to working towards a solution to this issue. Attached below are all of the documents you requested, post the changes you instructed me to preform. The only catch is, when I attempt to upload my allPEs.log.txt file, I am informed I am unable to upload it as it, "is not included in the list".

HEMCO.log.txt HEMCO_Config.rc.txt ExtData.rc.txt gchp_5467578.out.txt

lizziel commented 2 years ago

No worries, I wonder if allPEs.log is too large. Could you open it up and look at the end to see if there are any error messages? If there are, please copy them here along with any text right above it.

lizziel commented 2 years ago

Actually, I was able to figure out the issue by looking at the MAPL traceback in the log file. It is hitting an error in the file refresh template within this code section:

                        ! scaling entry                                                   
                        call ESMF_ConfigGetAttribute(CFtemp, c_scale, __RC__)
                        if (trim(c_scale) == "none") then
                           primary%item(totalPrimaryEntries)%do_scale = .false.
                        else
                           primary%item(totalPrimaryEntries)%do_scale = .true.
                           read(c_scale,*,iostat=ios) primary%item(totalPrimaryEntries)%s\
cale
                        end if

                        ! variable name on file entry                                     
                        call ESMF_ConfigGetAttribute(CFtemp, primary%item(totalPrimaryEnt\
ries)%var,    __RC__)
                        ! file template entry                                             
                        call ESMF_ConfigGetAttribute(CFtemp, primary%item(totalPrimaryEnt\
ries)%file,   __RC__)

This means that when MAPL is parsing ExtData.rc it cannot find the refresh template information about file time read frequency. Looking at your file, you have entries like this:

MEGAN-MACC_BIO_C2H6 kg/m2/s Y Y none none emiss_bio /scratch/ExtData/HEMCO/MEGAN-MACC_Biogenic_C2H6/MEGAN-MACC_Glb_0.5x0.5_bio_ethane__monthly.nc

Contrast that with columns in existing emissions, such as this: APEI_NO kg/m2/s N Y F%y4-01-01T00:00:00 none none NOx ./HcoDir/APEI/v2016-11/APEI.0.1x0.1.nc

That APEI_NO example has F%y4-01-01T00:00:00 as the fifth column. You will need to add something similar for your new emissions. What to specify for the template is dependent on the time contents of your file. If the file is time-invariant then you can simply use -. If the file is monthly climatology for one year only you can use F2011-%m2-01T00:00:00, replacing 2011 with whatever year your data is. If your data file contains both months and years then you can use F%y4-%m2-01T00:00:00.

Description of the columns and instructions on how to fill them are at the top of ExtData.rc. Here is that information for quick reference:

#--------|-------|------|------------|---------------|--------|-------|---------------------|
# Export |       |      |            |___ Refresh ___|____ Factors ___|___ External File ___|
# Name   | Units | Clim |Conservative| Time Template | Offset | Scale | Variable | Template |
#--------|-------|------|------------|---------------|--------|-------|----------|----------|
#
# Notes:
# Units should be in single quotes if they contain whitespace
# Climatology should be Y if the file contains monthly climatology; otherwise it should be N
# Conservative should be Y if units imply mass conservation dependency on regrid method (e.g. value per area)
# Refresh Time Template should be:
#    1. - if the file contains time-invariant constants
#    2. prefixed with F if no time interpolation between data reads (F = fixed)
#    3. 0 if data should be kept up-to-date at all times
#    4. 0:HHMMSS if data occurs in file with frequency HHMMSS and should be updated at that frequency
#    5. %y4-%m2-%d2T%h2:00:00, with any tokens replaced by appropriate constant time value, if multiple time
#       values are in the same file. Note that including an F prefix indicates the data should be treated
#       as constant until the next refresh time. Omission of the F prefix will time-interpolate between
#       the values.
# Minimize whitespace wherever possible to avoid surpassing the ESMF read buffer character limit,
# applicable only for uncommented lines.

We are aware it is confusing, particularly regarding the refresh time. But hopefully this helps clarify things.

JbourJabber commented 2 years ago

Hi again @lizziel , I just wanted to follow up your detailed response with a hearty thank you--but I am continuing to face issues after having made the recommended changes (unless I did something incorrectly).

To answer your previous question regarding the allPEs.log, here is what is at the end of the file:

`0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): Start 0000: CAP.EXTDATA: DEBUG: >> REFFTIME for : <> 0000: CAP.EXTDATA: DEBUG: >> Reference time: 1985-01-01 00:00:00 0000: CAP.EXTDATA: DEBUG: >> Frequency : 0000-00-01 00:00:00

. . . Identical lines repeat until:

0000: CAP.EXTDATA: INFO: ExtData could not satisfy import Etiope_C2H6_emiss_geo 0000: CAP.EXTDATA: INFO: ExtData could not satisfy import HOURLY_SCALFACT 0000: CAP.EXTDATA: INFO: ExtData could not satisfy import MEGAN-MACC_C2H6_emiss_bio 0000: CAP.EXTDATA: INFO: ExtData could not satisfy import RETRO_C2H6_emiss_ocean`

Attached are my updated files post my most recent attempt to run, with the exception of the HEMCO_Config.rc--as nothing was changed in this file. Thank you again for all of your help.

ExtData.rc.txt HEMCO.log.txt gchp_5468014.out.txt

lizziel commented 2 years ago

Hi @JbourJabber, it looks like there is a mismatch in the names used in HEMCO_Config.rc and ExtData.rc which is causing GCHP's MAPL library (which handles I/O) to complain that it can't find entries in ExtData.rc. Try changing the import names in ExtData.rc (first column) to match the names in HEMCO_Config.rc. For example, change Etiope_GEO_C2H6 to Etiope_C2H6_emiss_geo to match the name in this line in HEMCO_Config.rc:

0 Etiope_C2H6_emiss_geo $ROOT/Etiope_Geologic_C2H6/Total_GEOC2H6_Output_2018.nc C2H6_Output 2018/1-12/1-31/0-23 C xy kg/m2/s C2H6 - 1 1

Incidentally, I noticed you have species name next to the logicals for your new emissions at the top of HEMCO_Config.rc. Your settings look like this:

###############################################################################
### BEGIN SECTION EXTENSION SWITCHES
###############################################################################
# ExtNr ExtName                on/off  Species  Years avail.
0       Base                   : on    *
    --> MEGAN-MACC_Biogenic_C2H6 :     true  C2H6  # 1980-2000
    --> MEGANv2_Biogenic_C2H6  :       true  C2H6  # 2000
    --> POET_Oceanic_C2H6      :       false  C2H6  # 1990
    --> RETRO_Oceanic_C2H6     :       true  C2H6  # 1960-2000
    --> Etiope_Geologic_C2H6   :       true  C2H6  # 2018

I do not think that is necessary (@msulprizio, please correct me if I am wrong on that). These settings are merely to set a logical flag about whether to include emissions in the block later on. Entries in that block specify the species.

msulprizio commented 2 years ago

I noticed you have species name next to the logicals for your new emissions at the top of HEMCO_Config.rc.

I do not think that is necessary (@msulprizio, please correct me if I am wrong on that). These settings are merely to set a logical flag about whether to include emissions in the block later on. Entries in that block specify the species.

That's correct. Unless specific modifications have been made to your local code @JbourJabber then you don't need the species column there. As @lizziel said, the species that HEMCO should apply fluxes to is specified in the specific entries for that inventory defined further down in HEMCO_Config.rc. If you want to include them there for your reference, I would recommend placing the species names after the comment character #.

JbourJabber commented 2 years ago

Thank you both very much, as the name correction and commenting of the species name did the trick in terms of preventing continued crashes at the same phase. Unfortunately, I am now dealing with some kind of segmentation fault (SIGSEGV ERROR) having to do with an invalid memory reference.

Attached below is a copy of the output.log, I am pretty sure the problem has to due with the following line:

'mpiexec noticed that process rank 9 with PID 0 on node compute099 exited on signal 11 (Segmentation fault).'

In the case it is, what would either of you recommend me change to mitigate continued failures?

gchp_5471879.out.txt

lizziel commented 2 years ago

Hi @JbourJabber, is there anything indicating an error in log file allPEs.log? You can also try building the model with debug flags on by using -DCMAKE_BUILD_TYPE=Debug in the cmake command and then rerunning. This may catch the issue. See also GCHP ReadTheDocs section on build options.

JbourJabber commented 2 years ago

Hi @lizziel, I apologize for the delay. To answer your question, no I cannot seemingly locate any error in the allPEs.log. Here is the last section of said log file:

0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A1.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.I3.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.I3.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.I3.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.I3.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.I3.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.I3.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3cld.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3cld.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3cld.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3cld.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3cld.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3cld.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3dyn.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3dyn.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3dyn.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3dyn.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3mstC.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3mstC.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3mstC.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3mstC.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3mstE.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3mstE.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3mstE.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3mstE.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2018/01/MERRA2.20180101.A3mstE.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2015/01/MERRA2.20150101.CN.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2015/01/MERRA2.20150101.CN.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2015/01/MERRA2.20150101.CN.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2015/01/MERRA2.20150101.CN.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2015/01/MERRA2.20150101.CN.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2015/01/MERRA2.20150101.CN.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2015/01/MERRA2.20150101.CN.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2015/01/MERRA2.20150101.CN.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2015/01/MERRA2.20150101.CN.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./MetDir/2015/01/MERRA2.20150101.CN.05x0625.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./HcoDir/OFFLINE_LIGHTNING/v2020-03/MERRA2/2018/FLASH_CTH_MERRA2_0.5x0.625_2018_01.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./HcoDir/OFFLINE_LIGHTNING/v2020-03/MERRA2/2018/FLASH_CTH_MERRA2_0.5x0.625_2018_01.nc4 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./HcoDir/Yuan_XLAI/v2019-03/Condensed_Yuan_proc_MODIS_XLAI.025x025.2005.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./HcoDir/MEGAN-MACC_Biogenic_C2H6/MEGAN-MACC_Glb_0.5x0.5_bio_ethane__monthly.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./HcoDir/MEGAN-MACC_Biogenic_C2H6/MEGAN-MACC_Glb_0.5x0.5_bio_ethane__monthly.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop: 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./HcoDir/RETRO_Oceanic_C2H6/RETRO_Glb_0.5x0.5_oce_ETHANE__monthly.nc 0000: CAP.EXTDATA: DEBUG: Retrieving formatter for: ./HcoDir/RETRO_Oceanic_C2H6/RETRO_Glb_0.5x0.5_oce_ETHANE__monthly.nc 0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop:

I will continue troubleshooting with the rebuilt model debugging flags you recommended.

lizziel commented 2 years ago

It looks like it fails at or right after getting information about the RETRO file. This is what you have as an entry for RETRO: RETRO_OCE_C2H6 kg/m2/s Y Y F%y4-%m2-01T00:00:00 none none ocean ./HcoDir/RETRO_Oceanic_C2H6/RETRO_Glb_0.5x0.5_oce_ETHANE__monthly.nc

The first Y indicates the file is climatology but this is in conflict with giving the %y4 token for variable year. If your file has 12 months of data for one year change the %y4 to the year in the file. If there are multiple years in the file change the first Y to F. Hopefully that will then get past the error.

JbourJabber commented 2 years ago

I just wanted to follow up and confirm that making the changes recommended unfortunately made no difference--but it did lead to a few more questions:

1st: Should the change be from Y to F or Y to N? In the ExtData.rc file, it states the following:

# Climatology should be Y if the file contains monthly climatology; otherwise it should be N

2nd: In the allPE's.log, it appears that the MEGAN-MACC_Biogenic_C2H6 inventory has the identical schema compared to the RETRO_Oceanic_C2H6 and is successfully initialized? See below:

MEGAN-MACC_C2H6_emiss_bio kg/m2/s F Y F%y4-%m2-01T00:00:00 none none emiss_bio ./HcoDir/MEGAN-MACC_Biogenic_C2H6/MEGAN-MACC_Glb_0.5x0.5_bio_ethane__monthly.nc

RETRO_C2H6_emiss_ocean kg/m2/s F Y F%y4-%m2-01T00:00:00 none none ocean ./HcoDir/RETRO_Oceanic_C2H6/RETRO_Glb_0.5x0.5_oce_ETHANE__monthly.nc

3rd: What impact would a discrepancy between the available years of the emission inventory and the desired simulation year have? For example:

None of my additional emission inventories possess data for the year 2018, which is the desired simulation year--would this be known to cause or be related to any of the issues faced?

lizziel commented 2 years ago

HI @JbourJabber,

For the climatology, if your file contains 12 months of data for one year only then it can be considered climatology in ExtData for input purposes. You would then want "Y Y" rather than "F Y" assuming you want conservative regridding (which we recommend). If you have multiple years of monthly data in the file then it would not be climatology and you should use "F T".

If your data is for years outside of the simulation period then the nearest available year will be used. This assumes the extrapolate year functionality is enabled at the top of ExtData.rc (see here). It is always enabled by default in GCHP.

Could you post your log and config files for your latest run?

JbourJabber commented 2 years ago

Hi @lizziel,

Hope you're well. Attached below are the log and config files from my latest run. Still no luck getting the job to run properly, but there have been some developments over the last couple of weeks. First, here are those files:

  1. GCHP.rc.txt
  2. HISTORY.rc.txt
  3. PET000.ESMF_LogFile.txt
  4. gchp_5525821.out.txt
  5. HEMCO.log (2).txt
  6. logging.yml.txt
  7. HEMCO_Diagn.rc.txt
  8. gchp.runscript.txt
  9. runConfig.sh.txt
  10. HEMCO_Config.rc.txt
  11. Last Dozen Lines of allPEs.log.txt: 
    0000: CAP.EXTDATA: DEBUG:  Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc
0000: CAP.EXTDATA: DEBUG:  Retrieving formatter for: ./ChemDir/Olson_Land_Map_201203/Olson_2001_Land_Map.025x025.generic.nc
0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop:
0000: CAP.EXTDATA: DEBUG:  Retrieving formatter for: ./HcoDir/Yuan_XLAI/v2019-03/Condensed_Yuan_proc_MODIS_XLAI.025x025.2005.nc
0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop:
0000: CAP.EXTDATA: DEBUG:  Retrieving formatter for: ./HcoDir/MEGAN-MACC_Biogenic_C2H6/MEGAN-MACC_Glb_0.5x0.5_bio_ethane__monthly_2018.nc
0000: CAP.EXTDATA: DEBUG:  Retrieving formatter for: ./HcoDir/MEGAN-MACC_Biogenic_C2H6/MEGAN-MACC_Glb_0.5x0.5_bio_ethane__monthly_2018.nc
0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop:
0000: CAP.EXTDATA: DEBUG:  Retrieving formatter for: ./HcoDir/RETRO_Oceanic_C2H6/RETRO_Glb_0.5x0.5_oce_ETHANE__monthly_2000.nc
0000: CAP.EXTDATA: DEBUG:  Retrieving formatter for: ./HcoDir/RETRO_Oceanic_C2H6/RETRO_Glb_0.5x0.5_oce_ETHANE__monthly_2000.nc
0000: CAP.EXTDATA: DEBUG: ExtData Initialize_(): PrimaryLoop:

At present time, with your assistence and countless hours of troubleshooting--I've successfully gotten the MEGAN-MACC Biogenic C2H6 and RETRO Oceanic C2H6 Emission Inventories working in HEMCO. Where the crash continues to appear to crash is when initializing my custom Geologic C2H6 Emission Inventory. This inventory was constructed from .csv files and aggregated into a netCDF using several python modules, see script below:

# Fetches internal data of target .csv (Excel) or .xlsx (Excel) files and writes to .nc (NetCDF)
# By doing so, this will allow for the import of key climate variable data directly into HEMCO within GEOS-Chem GCHP

# Key Module Import(s)
import numpy as np
import netCDF4 as nc
import pandas as pd
import xarray as xr
from datetime import datetime

# Key Variable(s)
GEO_C2H6_Dir = ('E:/PSU/CCAR REU/Emission(s) Inventories (GEOS-CHEM)/Simulation Data/Geo-CH4_emission_grid_files (Geologic)/Gridded Geologic C2H6 - Emissions Inventory/') # Geologic C2H6 Data Directory

# Location of .csv file to be Loaded into Python
GEO = (GEO_C2H6_Dir + 'Total_geoC2H6_output_2018_Annual.csv') # Location of Global Geologic C2H6 Emission Data

# Read .csv file into Python via Pandas Series
s1 = pd.read_csv(GEO, sep = ",", header = 'infer', parse_dates = ['Time'],  usecols = ['Lon', 'Lat', 'Time', 'Area', 'Out_tonnes_per_year_by_cell', 'C2H6_Output'], 
                 dtype = {'Lon':np.float32, 'Lat':np.float32, 'Area':np.float32, 'Out_tonnes_per_year_by_cell':np.float32, 'C2H6_Output':np.float32})

# Drop NaN Values in Pandas Series
s1.dropna(subset = ['Time'])

# Convert String Timeseries to Datetime Variable using Datetime Module
pd.to_datetime(s1['Time'], format = '%m/%d/%Y')

# Naming convention for Established Pandas Series
s1.name = 'GEOC2H6_Total'

# Set Index/Dimension(s) of the Impending Xarray Dataframe
df1 = (s1.set_index(['Lat', 'Lon', 'Time']).to_xarray())

# Create Xarray Dataframe from Pandas Series
xds1 = xr.Dataset(df1)

# Assignment of Variable & Attribute Metadata
xds1['Lon'].attrs = {'long_name':'Longitude', 'units':'degrees_east', 'axis':'X'}
xds1['Lon'].encoding = {'dtype':'float32', '_FillValue': None}
xds1['Lat'].attrs = {'long_name':'Latitude','units':'degrees_north', 'axis':'Y'}
xds1['Lat'].encoding = {'dtype':'float32', '_FillValue': None}
#xds1['Lev'].attrs = {'long_name':'GEOS-Chem level', 'units':'level', 'axis':'Z', 'positive':'up'}
xds1['Time'].attrs = {'long_name':'Time', 'axis':'T'}
xds1['Time'].encoding = {'units':'hours since 2018-01-01 00:00:00', 'calendar':'standard', '_FillValue': None}
xds1['Area'].attrs = {'Long_Name':'Area of each Grid Cell in meters-squared', 'units':'m-2', '_ChunkSizes':[1, 45, 360]}
xds1['Area'].encoding = {'dtype':'float32', '_FillValue': None}
xds1['Out_tonnes_per_year_by_cell'].attrs = {'units':'Tonnes per Gridcell', 'Long_Name':'Total Output of Cumulative Ethane in units of Tonnes per Year in each Cell', '_ChunkSizes':[1, 45, 360]}
xds1['Out_tonnes_per_year_by_cell'].encoding = {'dtype':'float32', '_FillValue': None}
xds1['C2H6_Output'].attrs = {'units':'kg m-2 s-1', 'Long_Name':'Total Output of Cumulative Ethane in units of kilograms of C2H6 per m^2 per sec', '_ChunkSizes':[1, 45, 360]}
xds1['C2H6_Output'].encoding = {'dtype':'float32', '_FillValue': None}

# Addition of Global Attribute Metadata
xds1.attrs = {'Title':'Total_GEOC2H6_Output_2018', 'Dataset PI':'Jordan-Yoosuf Al-Jbour', 'Original Files':'Total_geoCH4_output_2018.csv, Total_geoCH4_13C_2018.csv, SS_output_2018.csv, SS_13C_2018.csv, OS_output_2018.csv, OS_13C_2018.csv, MS_output_2018.csv, MS_13C_2018.csv, GM_output_2018.csv, GM_13C_2018.csv',
              'References':'Etiope, G., Ciotoli, G., Schwietzke, S., and Schoell, M.: Global geological CH4 emission grid files. https://doi.org/10.25925/4J3F-HE27, 2018. Etiope, G., Ciotoli, G., Schwietzke, S., and Schoell, M.: Gridded maps of geological methane emissions and their isotopic signature, Earth Syst. Sci. Data, 11, 1-22, 2019 https://doi.org/10.5194/essd-11-1-2019', 
              'Summary':'Total output of Cumulative Geologic Emitted Ethane Ratiod from Cumulative Geologic Emitted Methane from 2018.'
              , 'Source':'Driving variables are: landcover, weather, and atmospheric chemical composition.', 'Institute':'Portland State University in Portland, OR 97202 - Center for Climate and Aerosol Research'
              , 'nco_openmp_thread_number':1, 'Conventions':'CF-1.0', 'Distribution Date':'July 30th, 2021'}

# Save to Output NetCDF
xds1.to_netcdf(path = 'E:/PSU/CCAR REU/Emission(s) Inventories (GEOS-CHEM)/Simulation Data/Geo-CH4_emission_grid_files (Geologic)/Gridded Geologic C2H6 - Emissions Inventory/Total_GEOC2H6_Output_2018_4th_attempt.nc')

Though I followed these pages closely, I have a feeling that this netCDF is not meeting the isCoards standard:

  1. http://wiki.seas.harvard.edu/geos-chem/index.php/Working_with_netCDF_data_files#Edit_variable_names_and_attributes
  2. http://wiki.seas.harvard.edu/geos-chem/index.php/The_COARDS_netCDF_conventions_for_earth_science_data#COARDS_Global_attributes
  3. http://wiki.seas.harvard.edu/geos-chem/index.php/Preparing_data_files_for_use_with_HEMCO#Overview
  4. http://wiki.seas.harvard.edu/geos-chem/index.php/The_HEMCO_User%27s_Guide#Base_emissions

Here is the Metadata for the problematic netCDF:

netcdf file:/E:/PSU/CCAR%20REU/Emission(s)%20Inventories%20(GEOS-CHEM)/Simulation%20Data/Geo-CH4_emission_grid_files%20(Geologic)/Gridded%20Geologic%20C2H6%20-%20Emissions%20Inventory/Total_GEOC2H6_Output_2018_4th_attempt.nc {
  dimensions:
    Lat = 180;
    Lon = 360;
    Time = 12;
  variables:
    float Lat(Lat=180);
      :long_name = "Latitude";
      :units = "degrees_north";
      :axis = "Y";

    float Lon(Lon=360);
      :long_name = "Longitude";
      :units = "degrees_east";
      :axis = "X";

    long Time(Time=12);
      :long_name = "Time";
      :units = "hours since 2018-01-01";
      :calendar = "standard";
      :axis = "T";

    float Out_tonnes_per_year_by_cell(Lat=180, Lon=360, Time=12);
      :units = "Tonnes per Gridcell";
      :Long_Name = "Total Output of Cumulative Ethane in units of Tonnes per Year in each Cell";
      :_ChunkSizes = 1, 45, 360; // int

    float Area(Lat=180, Lon=360, Time=12);
      :Long_Name = "Area of each Grid Cell in meters-squared";
      :units = "m-2";
      :_ChunkSizes = 1, 45, 360; // int

    float C2H6_Output(Lat=180, Lon=360, Time=12);
      :units = "kg m-2 s-1";
      :Long_Name = "Total Output of Cumulative Ethane in units of kilograms of C2H6 per m^2 per sec";
      :_ChunkSizes = 1, 45, 360; // int

  // global attributes:
  :Title = "Total_GEOC2H6_Output_2018";
  :Dataset_PI = "Jordan-Yoosuf Al-Jbour";
  :Original_Files = "Total_geoCH4_output_2018.csv, Total_geoCH4_13C_2018.csv, SS_output_2018.csv, SS_13C_2018.csv, OS_output_2018.csv, OS_13C_2018.csv, MS_output_2018.csv, MS_13C_2018.csv, GM_output_2018.csv, GM_13C_2018.csv";
  :References = "Etiope, G., Ciotoli, G., Schwietzke, S., and Schoell, M.: Global geological CH4 emission grid files. https://doi.org/10.25925/4J3F-HE27, 2018. Etiope, G., Ciotoli, G., Schwietzke, S., and Schoell, M.: Gridded maps of geological methane emissions and their isotopic signature, Earth Syst. Sci. Data, 11, 1-22, 2019 https://doi.org/10.5194/essd-11-1-2019";
  :Summary = "Total output of Cumulative Geologic Emitted Ethane Ratiod from Cumulative Geologic Emitted Methane from 2018.";
  :Source = "Driving variables are: landcover, weather, and atmospheric chemical composition.";
  :Institute = "Portland State University in Portland, OR 97202 - Center for Climate and Aerosol Research";
  :nco_openmp_thread_number = 1; // int
  :Conventions = "CF-1.0";
  :Distribution_Date = "July 30th, 2021";}

The netCDF is functional and projectable within Panoply, but I believe is the source of the current issues faced with getting this run to work within GEOS-Chem, see below:

C2H6_Output_in_Total_GEOC2H6_Output_2018

Looking forward to hearing back and kind regards.

JbourJabber commented 2 years ago

I also wanted to provide the results of the isCoards verification script:

===========================================================================
Filename: /scratch/ExtData/HEMCO/Etiope_Geologic_C2H6/Total_GEOC2H6_Output_2018.nc
===========================================================================

The following items adhere to the COARDS standard:
---------------------------------------------------------------------------
-> Dimension "Lat" adheres to standard usage
-> Dimension "Lon" adheres to standard usage
-> Dimension "Time" adheres to standard usage
-> Lat(Lat)
-> Lat is monotonically increasing
-> Lat:axis = "Y"
-> Lat:long_name = "Latitude"
-> Lat:units = "degrees_north"
-> Lon(Lon)
-> Lon is monotonically increasing
-> Lon:axis = "X"
-> Lon:long_name = "Longitude"
-> Lon:units = "degrees_east"
-> Time(Time)
-> Time is monotonically increasing
-> Time[0] = 0 (this is required for GCHP)
-> Time:axis = "T"
-> Time:calendar = "standard"
-> Time:long_name = "Time"
-> Time:units = "days since 2018-01-15T00:00:00+00:00"
-> Out_tonnes_per_year_by_cell(Lat,Lon,Time)
-> Out_tonnes_per_year_by_cell:long_name = "Total Output of Cumulative Ethane in units of Tonnes per Year in each Cell"
-> Out_tonnes_per_year_by_cell:units = "Tonnes per Gridcell"
-> Area(Lat,Lon,Time)
-> Area:long_name = "Area of each Grid Cell in meters-squared"
-> Area:units = "m-2"
-> C2H6_Output(Lat,Lon,Time)
-> C2H6_Output:long_name = "Total Output of Cumulative Ethane in units of kilograms of C2H6 per m^2 per sec"
-> C2H6_Output:units = "kg m-2 s-1"
-> conventions: "CF-1.0"
-> history: "Processed for Aljbour et al, 2022"
-> title: "Total_GEOC2H6_Output_2018"
-> format: "netCDF (nc4)"
-> references: "Etiope, G., Ciotoli, G., Schwietzke, S., and Schoell, M.: Global geological CH4 emission grid files. https://doi.org/10.25925/4J3F-HE27, 2018. Etiope, G., Ciotoli, G., Schwietzke, S., and Schoell, M.: Gridded maps of geological methane emissions and their isotopic signature, Earth Syst. Sci. Data, 11, 1-22, 2019 https://doi.org/10.5194/essd-11-1-2019"

The following items DO NOT ADHERE to the COARDS standard:
---------------------------------------------------------------------------

The following optional items are RECOMMENDED:
---------------------------------------------------------------------------
-> Consider adding Out_tonnes_per_year_by_cell:_FillValue
-> Consider adding Out_tonnes_per_year_by_cell:missing_value
-> Consider adding Area:_FillValue
-> Consider adding Area:missing_value
-> Consider adding C2H6_Output:_FillValue
-> Consider adding C2H6_Output:missing_value
lizziel commented 2 years ago

Hi @JbourJabber, could you also provide your latest ExtData.rc file?

JbourJabber commented 2 years ago

Hi @lizziel! I am very excited to announce that I figured out what the issue was and I've been able to get several simulations running at different resolutions. All after making the following changes to the how the netCDF was written:

Solution that was found to Total_GEOC2H6_Output_2018.nc:

  1. Change in .csv file header: change order of dimension variables from 'lon', 'lat', 'time' --> 'lat', 'lon', 'time'
  2. Change in .csv file header: change all upper-case characters in dimension variables from 'Lat', 'Lon', 'Time' --> 'lat', 'lon', 'time'
  3. Change at all relevant lines of Script: change all upper-case characters in dimension variables from 'Lat', 'Lon', 'Time' --> 'lat', 'lon', 'time'
  4. Change at Line #33 of Script: pd.to_datetime(s1['time'], format = '%m/%d/%Y') --> pd.to_datetime(s1['time'], format = '%m/%d/%Y %H:%M')
  5. Change at Line #39 of Script: df1 = (s1.set_index(['lon','lat', 'time']).to_xarray()) --> df1 = (s1.set_index(['time','lat', 'lon']).to_xarray())
  6. Change at all relevant lines of script: change order of attribute assignment and encoding, from Lat', 'Lon', 'Time' --> 'time', 'lat', 'lon'
  7. Removal of the '_ChunkSizes' component of attributes for 'Area', 'Out_tonnes_per_year_by_cell', 'C2H6_Output'.

I just want to thank you and the rest of the GEOS-Chem Team for all of your help troubleshooting these issues over the past few months. It has been deeply appreciated.

Kind regards and best of luck!