search

USEPA / CMAQ

Code for U.S. EPA’s Community Multiscale Air Quality Model (CMAQ) which helps in conducting air quality model simulations
https://epa.gov/CMAQ
MIT License
280 stars 201 forks source link

error - related to I/O API MXFILE3 = 64 open file limit when testing CMAQ_DDM #86

Closed lizadams closed 5 years ago

lizadams commented 7 years ago

Obtained this error when testing a run of CMAQ v5.2 DDM at UNC Compiler: gcc (on UNC systems used command module load openmpi_gcc/4.8.1) I/O API version: 3.1

 >>--->> WARNING in subroutine OPEN3     Could not open S_CGRID         Maximum number of files already have been opened.
     *** ERROR ABORT in subroutine WR_CGRID on PE 000
     Could not open S_CGRID file
     Date and time 0:00:00   July 2, 2011   (2011183:000000)
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 32767.

Work around: Reduced the number of files opened by changing the following settings in the run script. (These settings also matched the run script run.ddm.cmas provided by Sergei)

setenv CTM_ABFLUX N
setenv CTM_GRAV_SETL N

Proposed fix: Recompile I/O API with a larger MXFILE3 setting

bnmurphy commented 7 years ago

Sounds good Liz. Then do we have to tell every user to do this as well?

lizadams commented 7 years ago

Yes - if they obtain the same error, this would be a workaround. It may be too much to ask for folks to recompile the ioapi library. I haven't tried this myself.

jslinvin commented 6 years ago

A DDM run I set up returned the same error "Could not open A_SENS_1 Maximum number of files already have been opened." Neither the workaround nor recompiling ioapi would fix the problem. MXFILES3 has been increased to 64, 256, and 1024 but the same error persisted. What other changes need to be made in order to fix the problem? Also, is there a way to know the number of files ioapi has open?

compiler: pgi version 16.9, ioapi 3.1, netcdf 3.6.3. Regular CMAQ5.2 run (without DDM) and DDM with CMAQ5.0.2 did not have this problem with "max files open".

Jin

bnmurphy commented 6 years ago

@jslinvin I'm not aware of any way to ask IOAPI how many files are open although using the JUNIT() function might help. I'm not sure this issue will be resolved in CMAQv5.3, unless Carlie can weigh in.

kmfoley commented 5 years ago

There is a now an I/O API version 3.2 "large" that is designed for applications with a large number of model output files (e.g. utilizing all of CMAQ's optional diagnostic output files) and/or a large number of model variables (e.g. CMAQ-HDDM or CMAQ-ISAM applications). I/O API v3.2-large increases the MXFILE3 variable from 64 to 512 and increases the MXVARS3 varialbe from 2048 to 16384. This version is available as a zip file from the following address: https://www.cmascenter.org/ioapi/download/ioapi-3.2-large.tar.gz

Installation instructions for I/O API v5.3-large are provided in README.txt in the .tar.gz file.