NOAA-OWP / snow17

Other
4 stars 10 forks source link

Limitation on number of open files for standalone version of Snow-17 #45

Closed drakest123 closed 2 months ago

drakest123 commented 2 months ago

Multiple catchments can be processed during a single execution of the standalone version of Snow-17. However, the program will crash when there is a conflict with an open file unit.

Current behavior

The program crashes when concurrently processing 44 or more catchments.

Expected behavior

There should be a check for open unit numbers and an error check if the user tries to process more than an established number of catchments simultaneously.

Considerations

  1. Snow-17 runs within NGEN execute a single catchment at a time. This method processes each catchment, however, it is slow and does not support code optimization to process many catchments simultaneously.
  2. The number of input forcing files could become unwieldy if there is a strict one-to-one correspondence between forcing files and catchments.
  3. The Fortran specs indicate the one can open as few as 100 files at a time (file units 0-99) and some of these file units are dedicated to standard I/O (e.g. unit numbers 4,5,6). The number of files one can open at a time is implementation specific.
  4. One reference indicated that you can change #define MXUNIT 100 and recompile the Fortran compiler to increase the number of files that can be opened at a time. However, there is also an OS limitation. On this Mac M1 the limitation is 256:
    % ulimit -n
    256

Suggested solutions

  1. Write a function/subroutine to check for open unit numbers and dole out an unused unit number starting at 10 to avoid the OS range. Avoid unit numbers that are explicitly utilized by the program (e.g., 51) or change the code such that all unit numbers are doled out.
  2. Error check for requests that exceed an established number of catchments, e.g., 80.
drakest123 commented 2 months ago

Moved to previous issue.