search

metno / emep-ctm

Open Source EMEP/MSC-W model
GNU General Public License v3.0
27 stars 18 forks source link

config_emep.nml Fortran runtime error: End of file #44

Closed JessMBemep closed 5 years ago

JessMBemep commented 5 years ago

Hi, I'm trying to recreate the base run of the software. The code appears to compile okay, at least with the default settings. But when I run modrun.sh, I'am getting an error: At line 728 of file Config_module.f90 (unit = 28, file = 'config_emep.nml') Fortran runtime error: End of file

the 728 line is: read(IO_NML,NML=ModelConstants_config)

I'm using gfortran compiler. I read that some fortran executables can be sensitive to line endings, demanding the correct line-terminator for the OS, including on the last line of a data file. However in your code you specifically comment to "! do not close(IO_NML), other modules will be read namelist on this file"

It must be a very stupid error, but I'm affraid to change a code line which could generate further problems. Could you please advise about it?

Many thanks in advance

avaldebe commented 5 years ago

The gFortran compiler does not like comments inside a namelist. Try removing the text after the ! characters. For more specific suggestions I would need to see your configuration file.

JessMBemep commented 5 years ago

I removed all comments but does not work. Please, see attached the config_emep.nml file

JessMBemep commented 5 years ago 
&INPUT_PARA
  GRID      = 'EECCA',
  iyr_trend = 2015
  runlabel1 = 'Base',
  runlabel2 = 'Opensource_Setup_2018',
  startdate = 2015,01,01,00,
  enddate   = 2015,12,31,24,
&end
&Machine_config
  DataPath(1) = '../input',
&end
&ModelConstants_config
  meteo                 = '../meteoYYYY/GRID/meteoYYYYMMDD.nc',
  DegreeDayFactorsFile  = 'MetDir/DegreeDayFactors.nc',
  EmisHeightsFile       = 'DataDir/EmisHeights.txt',
  SplitDefaultFile      = 'DataDir/emissplit.defaults.POLL',
  SplitSpecialsFile     = 'DataDir/emissplit.specials.POLL',
  MonthlyFacFile        = 'DataDir/MonthlyFac.POLL',
  DailyFacFile          = 'DataDir/DailyFac.POLL',
  HourlyFacFile         = 'DataDir/HourlyFacs.INERIS',
  EMEP_EuroBVOCFile     = 'DataDir/EMEP_EuroBVOC.nc',
  EXP_NAME              = 'EMEPSTD',
  MY_OUTPUTS            = 'EMEPSTD',
  EmBio%GlobBvocMethod  = 'GLC-CLM',
  EmBio%CLF             =  0.59,
  LandCoverInputs%LandDefs  = 'DataDir/Inputs_LandDefs.csv',
  LandCoverInputs%Do3seDefs = 'DataDir/Inputs_DO3SE.csv',
  LandCoverInputs%MapFile   = 'DataDir/Landuse_PS_5km_LC.nc',
                              'DataDir/glc2000xCLMf18.nc',
  YieldModifications    = 'VBS',
  PBL%ZiMIN             =  50.0,
  USES%testname         = 'EMEPSTD',
  USES%FOREST_FIRES     = F,
  USES%EMISSTACKS       = F,
  USES%DEGREEDAY_FACTORS = T,
  USES%PlumeMethod      = 'PVDI',
  USES%PFT_MAPS         = F,
  USES%SOILWATER         = T,
  USES%CONVECTION        = F,
  USES%SEASALT           = T,
  USES%AIRCRAFT_EMIS     = T,
  USES%LIGHTNING_EMIS    = T,
  USES%EURO_SOILNOX      = T,
  EURO_SOILNOX_DEPSCALE = 1.0,
  USES%GLOBAL_SOILNOX    = F,
  USES%POLLEN            = F,
  USES%ROADDUST          = T,
  USES%DUST              = T,
  NETCDF_DEFLATE_LEVEL  = 4,
  INERIS_SNAP1          = F,
  INERIS_SNAP2          = F,
  EmisDir               = 'DataDir/EMEP01',
  emis_inputlist(1)%name= 'EmisDir/GNFRemis_EMEP01_YYYY.nc',
  emis_inputlist(2)%name= 'DataDir/OceanicEmissions_GEIA.nc',
  emis_inputlist(2)%type= 'DMS',
  emis_inputlist(3)%name= 'DataDir/FMIGlobShip2015mon.nc',
  emis_inputlist(3)%type= 'Special_ShipEmis',
  EMIS_OUT              = F,
  SELECT_LEVELS_HOURLY  = F,
  FORECAST              = F,
  USES%ASH               = T,
  ANALYSIS              = F,
  USES%AOD               = F,
  BGND_CH4              = -1,
  SKIP_RCT              = -1,
  FLUX_VEGS = 'IAM_CR','IAM_DF','IAM_MF',
  FLUX_IGNORE = 'W', 'D', 'DE', 'ICE', 'BARE'
&end
&ColumnSource_config
  flocdef='DataDir/columnsource_location.csv',
  femsdef='DataDir/columnsource_emission.csv',
  need_topo=T,
  NMAX_LOC=-1,
  NMAX_EMS=-1,
&end
&Fire_config
  need_poll = F,
  FINN_PATTERN='../input/FFire_YYYY.nc',
&end
&Nest_config
  MODE_READ = 'NONE',
  MODE_SAVE = 'NONE',
  template_read_3D = 'EMEP_IN.nc',
  template_read_BC = 'EMEP_IN.nc',
  template_write   = 'EMEP_OUT.nc',
&end
&ExternalBICs_config
  USE_EXTERNAL_BIC  = F,
  EXTERNAL_BIC_NAME = 'DUMMY',
&end
&OutputConcs_config
OutputConcs=
  'HMIX'                ,'m'   ,'2d','HMIX'     ,'MISC' ,'YM',
  'T2m'                 ,'degC','2d','T2m'      ,'MISC' ,'YMD',
  'SURF_ppbC_VOC'       ,'ppb' ,'2d','VOC'      ,'MISC' ,'YMD',
  'SO2'                 ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'NH3'                 ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'HNO3'                ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'NO2'                 ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'NO'                  ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'SO4'                 ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'NO3_F'               ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'NO3_C'               ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'NH4_F'               ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'SEASALT_F'           ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'SEASALT_C'           ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'DUST_ROAD_F'         ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'DUST_ROAD_C'         ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'DUST_WB_F'           ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'DUST_WB_C'           ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'DUST_SAH_F'          ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'DUST_SAH_C'          ,'ug'  ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'O3'                  ,'ppb' ,'2d','AIR_CONCS','SPEC' ,'YMDI',
  'NO'                  ,'ppb' ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'NO2'                 ,'ppb' ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'NH3'                 ,'ppb' ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'HNO3'                ,'ppb' ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'SO2'                 ,'ppb' ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'CO'                  ,'ppb' ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'HCHO'                ,'ppb' ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'C5H8'                ,'ppb' ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'OXN'                 ,'ugN' ,'2d','AIR_CONCS','GROUP','YMD',
  'NOX'                 ,'ugN' ,'2d','AIR_CONCS','GROUP','YMD',
  'RDN'                 ,'ugN' ,'2d','AIR_CONCS','GROUP','YMD',
  'TNO3'                ,'ug'  ,'2d','AIR_CONCS','GROUP','YMD',
  'SIA'                 ,'ug'  ,'2d','AIR_CONCS','GROUP','YMD',
  'PMFINE'              ,'ug'  ,'2d','AIR_CONCS','GROUP','YMD',
  'PM10'                ,'ug'  ,'2d','AIR_CONCS','GROUP','YMD',
  'PMCO'                ,'ug'  ,'2d','AIR_CONCS','GROUP','YMD',
  'PPM25'               ,'ug'  ,'2d','AIR_CONCS','GROUP','YMD',
  'PPM_C'               ,'ug'  ,'2d','AIR_CONCS','GROUP','YMD',
  'SS'                  ,'ug'  ,'2d','AIR_CONCS','GROUP','YMD',
  'DUST_NAT_F'          ,'ug'  ,'2d','AIR_CONCS','GROUP','YMD',
  'DUST_NAT_C'          ,'ug'  ,'2d','AIR_CONCS','GROUP','YMD',
  'DUST'                ,'ug'  ,'2d','AIR_CONCS','GROUP','YMD',
  'ECFINE'              ,'ug'  ,'2d','AIR_CONCS','GROUP','YMD',
  'ECCOARSE'            ,'ug'  ,'2d','AIR_CONCS','GROUP','YMD',
  'PMFINE'            ,'ug_PM' ,'2d','AIR_CONCS','GROUP','YMD',
  'OM25'              ,'ug_PM' ,'2d','AIR_CONCS','GROUP','YMD',
  'OM25'              ,'ugC_PM','2d','AIR_CONCS','GROUP','YMD',
  'OMCOARSE'          ,'ug_PM' ,'2d','AIR_CONCS','GROUP','YMD',
  'OMCOARSE'          ,'ugC_PM','2d','AIR_CONCS','GROUP','YMD',
  'ASOA'              ,'ug_PM' ,'2d','AIR_CONCS','GROUP','YMD',
  'BSOA'              ,'ug_PM' ,'2d','AIR_CONCS','GROUP','YMD',
  'BSOA'              ,'ug'    ,'2d','AIR_CONCS','GROUP','YMD',
  'PPM25_FIRE'        ,'ug'    ,'2d','AIR_CONCS','GROUP','YMD',
  'FFIRE_BC'          ,'ug'    ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'FFIRE_REMPPM25'    ,'ug'    ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'FFIRE_OM'          ,'ug'    ,'2d','AIR_CONCS','SPEC' ,'YMD',
  'AOD'                ,' '  ,'550nm','AOD:GROUP','MISC','YMD',
OutputVegO3=
  'POD1_IAM_DF'    ,'POD', 1.0,'MM','IAM_DF',F,0,999,'YM',
  'POD0_IAM_DF'    ,'POD', 0.0,'MM','IAM_DF',F,0,999,'YM',
  'POD1_IAM_MF'    ,'POD', 1.0,'MM','IAM_MF',F,0,999,'YM',
  'POD0_IAM_MF'    ,'POD', 0.0,'MM','IAM_DF',F,0,999,'YM',
  'POD1_DF'        ,'POD', 1.0,'MM','DF    ',F,0,999,'YM',
  'POD1_CF'        ,'POD', 1.0,'MM','CF    ',F,0,999,'YM',
  'POD3_TC'        ,'POD', 3.0,'MM','TC    ',F,0,999,'YM',
  'POD1_IAM_CR'    ,'POD', 1.0,'MM','IAM_CR',F,0,999,'YM',
  'POD0_IAM_CR'    ,'POD', 0.0,'MM','IAM_CR',F,0,999,'YM',
  'POD3_IAM_CR'    ,'POD', 3.0,'MM','IAM_CR',F,0,999,'YM',
  'MMAOT40_TC'     ,'AOT',40.0,'MM','TC    ',F,0,999,'YM',
  'MMAOT40_IAM_DF' ,'AOT',40.0,'MM','IAM_DF',F,0,999,'YM',
  'MMAOT40_IAM_MF' ,'AOT',40.0,'MM','IAM_MF',F,0,999,'YM',
  'MMAOT40_IAM_CR' ,'AOT',40.0,'MM','IAM_CR',F,0,999,'YM',
  'EUAOT40_Crops'  ,'AOT',40.0,'EU','IAM_CR',F,0,999,'YM',
  'EUAOT40_Forests','AOT',40.0,'EU','IAM_DF',F,0,999,'YM',
  'MMAOT40_IAM_WH' ,'AOT',40.0,'MM','IAM_WH',F,0,999,'YM',
&end
&OutputDep_config
DDEP_ECOS =
  'Grid'     ,'YMD',
  'Conif'    ,'YM',
  'Seminat'  ,'YM',
  'Water_D'  ,'Y',
  'Decid'    ,'Y',
  'Crops'    ,'Y',
DDEP_WANTED =
  'SOX'      ,'GROUP','mgS',
  'OXN'      ,'GROUP','mgN',
  'RDN'      ,'GROUP','mgN',
WDEP_WANTED =
  'PREC'     ,'PREC' ,'mm' ,'YMD',
  'SOX'      ,'GROUP','mgS','YMD',
  'OXN'      ,'GROUP','mgN','YMD',
  'RDN'      ,'GROUP','mgN','YMD',
  'SO2'      ,'SPEC' ,'mgS','YMD',
  'HNO3'     ,'SPEC' ,'mgN','YMD',
&end
&OutputSize_config
&end
avaldebe commented 5 years ago

make sure definition all lines end on a coma, e.g.

iyr_trend = 2015 FLUX_IGNORE = 'W', 'D', 'DE', 'ICE', 'BARE'

rosagierens commented 5 years ago

You could try ending each list with /, e.g. change 

&INPUT_PARA
  GRID      = 'EECCA',
  iyr_trend = 2015
  runlabel1 = 'Base',
  runlabel2 = 'Opensource_Setup_2018',
  startdate = 2015,01,01,00,
  enddate   = 2015,12,31,24,
&end

to 

&INPUT_PARA
  GRID      = 'EECCA',
  iyr_trend = 2015
  runlabel1 = 'Base',
  runlabel2 = 'Opensource_Setup_2018',
  startdate = 2015,01,01,00,
  enddate   = 2015,12,31,24,/

This helped me with the cray compiler.

JessMBemep commented 5 years ago

I reviewed all lines and included comas,I even removed spaces but do not work; I got the same error

rosagierens commented 5 years ago

Actually, I also replaced

  LandCoverInputs%MapFile   = 'DataDir/Landuse_PS_5km_LC.nc',
                              'DataDir/glc2000xCLMf18.nc',

with

  LandCoverInputs%MapFile(1)   = 'DataDir/Landuse_PS_5km_LC.nc',
  LandCoverInputs%MapFile(2)   = 'DataDir/glc2000xCLMf18.nc',
JessMBemep commented 5 years ago

Hi, I think there is another problem (or misundertanding from my side) around this error: Indeed, I did a lot of changes in the namelist in order to match with requierements of my gfortran version (4.8). They did not work. So that, I decided to explore other things, I realized that a copy of the namelist file (configemep.nml) was created in the work directory (i.e. the directory Base${GRID}_${NLEV}) from the modrun.sh launching. However this copy was empty. I have made the copy manually (from the terminal) and then the error ["Fortran runtime error... the 728 line is: read(IO_NML,NML=ModelConstants_config)"] dissapeared. So, it seems that the Config_module.f90 routine is reading the namelist. However, I got another issue for reading the namelist file (the same code):

 Found    20 MPI processes available
 Config_MC:NAMELIST START 
 Config_MC:WARNING: Could not find valid DataDir. Tried:
 ../input
mpiexec has exited due to process rank 0 with PID 0 on node lasigsrv2 exiting improperly.....

It seems a warnning in line 761 of Configmodule.f90; the code is indeed locating the input directory, I do not understand what is wrong. I'm wondering if my modrun.sh script is correctly customized? (see bellow). I'm suppossing that the Base${GRID}_${NLEV} directory is working directory, but actually I'm not sure. Could you please advice. Many thanks in advance

MODRUN.SH

#!/bin/bash

# Minimalistic script for run the Unified EMEP model
GRID=EECCA
NLEV=20lev
#cd ~/work/EMEP_MSC-W_model.rv4.17.OpenSource/Base_${GRID}_${NLEV}
cd /home/duruz/JMB/EMEP_MSC_W/EMEP_MSC-W_model.rv4.17.OpenSource/Base_${GRID}_${NLEV}

# Run the model
mpiexec /home/duruz/JMB/EMEP_MSC_W/emep-ctm-source/Unimod
avaldebe commented 5 years ago

The variable DataDir is set up on the Machine_config namelist

&Machine_config
  DataPath(1) = '../input',
&end

it indicates the path to most/all of the input files, e.g:

  EmisHeightsFile       = 'DataDir/EmisHeights.txt',
  SplitDefaultFile      = 'DataDir/emissplit.defaults.POLL',
  SplitSpecialsFile     = 'DataDir/emissplit.specials.POLL',
  MonthlyFacFile        = 'DataDir/MonthlyFac.POLL',

Following the setup shown on your modrun.sh, the full path the your input directory should be:

/home/duruz/JMB/EMEP_MSC_W/EMEP_MSC-W_model.rv4.17.OpenSource/input

If you placed the input files elsewhere, modify DataPath accordingly

JessMBemep commented 5 years ago

Yes, you are right. My input directory is placed at /home/duruz/JMB/EMEP_MSC_W/EMEP_MSC-W_model.rv4.17.OpenSource/input

But even when I write the full path: Found 20 MPI processes available Config_MC:NAMELIST START Config_MC:WARNING: Could not find valid DataDir. Tried: /home/duruz/JMB/EMEP_MSC_W/EMEP_MSC-W_model.rv4.17.OpenSource/input

avaldebe commented 5 years ago

Hi @gitpeterwind Can you help @JessMBemep with this issue. The error message is:

Config_MC:WARNING: Could not find valid DataDir.
JessMBemep commented 5 years ago

Just to provide you further details about the issue:

  1. The work directory that I declared in the modrun.sh: "/home/duruz/JMB/EMEP_MSC_W/EMEP_MSC-Wmodel.rv4.17.OpenSource/Base${GRID}_${NLEV}" is then created by the model or should I create it manually??? In my case, I do create it manually to avoid the error: ./modrun.sh: line 7: cd: /home/duruz/JMB/EMEP_MSC_W/EMEP_MSC-W_model.rv4.17.OpenSource/Base_EECCA_20lev: No such file or directory

  2. Once it exists, what is the script in charge of copying or linking the config_emep.nml into the Base_EECCA_20lev directory? In my case, when al launch modrun.sh, the config_emep.nml pass as an empty file into Base_EECCA_20lev directory, so I get the error: At line 728 of file Config_module.f90 (unit = 28, file = 'config_emep.nml') Fortran runtime error: End of file I do copy or link it manually for avoiding such error.

  3. Even so, there is something missing because I got the error: Config_MC:WARNING: Could not find valid DataDir. Tried: /home/duruz/JMB/EMEP_MSC_W/EMEP_MSC-W_model.rv4.17.OpenSource/input mpiexec has exited due to process rank 0 with PID 0 on node lasigsrv2 exiting improperly.....

gitpeterwind commented 5 years ago

Hi, Are you sure the directory actually exists (no syntax errors)? (test with "ls /home/duruz/JMB/EMEP_MSC_W/EMEP_MSC-W_model.rv4.17.OpenSource/input" )

Yes, the directory /home/duruz/JMB/EMEP_MSC_W/EMEP_MSC-W_model.rv4.17.OpenSource/Base_EECCA_20lev" must be created if it does not exist.

It seems all these problems are not really model related, and it is difficult for us to know the details of what you are doing. Perhaps you could ask some collegue which is used to general linux and mpi runs?

JessMBemep commented 5 years ago

Hi @gitpeterwind, Yes, the directory /home/duruz/JMB/EMEP_MSC_W/EMEP_MSC-W_model.rv4.17.OpenSource/input exists. the command line "ls /home/duruz/JMB/EMEP_MSC_W/EMEP_MSC-W_model.rv4.17.OpenSource/input" shows all data files and directories. I already ask other collegue and I also used to general linux. We have run other fortran based softwares like WRF and CHIMERE with the same machineand environment. Just to understand a litter bit more the model, Could you please answer my second question: What is the routine in charge of copying or linking the config_emep.nml into the Base_EECCA_20lev directory? In our case, the config_emep.nml is passing as an empty file into Base_EECCA_20lev directory. I do copy or link it manually (from the terminal) your answer could provide us some insights about the error Config_MC:WARNING: Could not find valid DataDir which is actually a warning of your fortran code Config_module.f90, lines 758-768:

  do i=1,size(DataPath)
    if(DataPath(i)=="NOTSET")then
      if(MasterProc)then
        write(*,*)dtxt//'WARNING: Could not find valid DataDir. Tried:'
        do j=1,i-1
          write(*,*)trim(DataPath(j))
        end do
        stop
      end if
      exit
    end if

I'm sorry for my insistence and many thanks in advance

gitpeterwind commented 5 years ago

Yes, you are right the config_emep.nml file should be copied or linked in the directory where the model is run. This is not clear form our "modrun.sh" example.

If you get the error message

"Config_MC:WARNING: Could not find valid DataDir. Tried: /home/duruz/JMB/EMEP_MSC_W/EMEP_MSC-W_model.rv4.17.OpenSource/input"

This means that the following fortran statement failed: "open(IO_TMP,file=trim(DataPath(i)),iostat=iostat,action='read')" Where "DataPath(i)" is the directory "/home/duruz/JMB/EMEP_MSC_W/EMEP_MSC-W_model.rv4.17.OpenSource/input"

It is difficult for me to say why this happens. It has been tested with gfortran too. You could uncomment the entire test in the code. It is just a test and does not do anything active. However if the directory is not accessible for some reason, the next read will probably fail.

!do i=1,size(DataPath)
!    if(DataPath(i)=="NOTSET")then
!      if(MasterProc)then
!   write(*,*)dtxt//'WARNING: Could not find valid DataDir. Tried:'
!        do j=1,i-1
!          write(*,*)trim(DataPath(j))
!        end do
!        stop
!      end if
!      exit
!    end if
!!   INQUIRE(...) does not behave consistently across intel/gfortran                                                                                  
!    open(IO_TMP,file=trim(DataPath(i)),iostat=iostat,action='read')! does not work without action='read'                                             
!    if(iostat==0)then
!      DataDir=trim(DataPath(i))
!      if(MasterProc)write(*,*)dtxt//'DataDir set to',trim(DataDir)
!      close(IO_TMP)
!      exit
!    end if
!  end do
JessMBemep commented 5 years ago

OK, thanks. I'll try Many thanks

gitpeterwind commented 5 years ago

Sorry I forgot that one line should be kept in the code to set DataDir (or you could even define directly your directory): DataDir=trim(DataPath(1))

JessMBemep commented 5 years ago

Hi, I continued having problems to access the input directory. I just uncommented the entire test in the code, and then, I created symbolic links of all files of the input directory into the source code directory. So, I do not use the DataDir path: &Machine_config DataPath(1) = '',

Also, instead of using the modrun.sh, I directly launch the executable from the source directory: mpiexec Unimod It works, and I have gotten the results of the base run.

Many thanks for your help

gitpeterwind commented 5 years ago

Thank you for the feedback. If you find out what went wrong in the original code, please let us know so that we can find a solution for possible other users with the same problem.