Connection error when importing pyschism submodules on a compute node

[mojy3995]

I ran generate_sourcesink_nc.py and it worked perfectly for 15 days simulation, but for a longer time (e.g. a year), I couldn't run it on a compute nodes based on the Jet policy for the memory usage, and I had to submit a job (sbatch). For a full year on Jet, I got connection error when I tried to import pyschism submodules on a compute node, because it doesn't have internet connection.

This part of the error messages indicates that the script was attempting to connect to a network resource, and failed.

"... /python3.9/urllib/request.py", line 1389, in https_open return self.do_open(http.client.HTTPSConnection, req, raise URLError(err) urllib.error.URLError: <urlopen error [Errno -2] Name or service not known> " Jet helpdesk suggested that somehow separate the computing and data gathering parts of our workflow, so that the compute nodes are occupied with tasks that require no additional downloading of anything. However, I have all input files downloaded in my directory and this is not about the data that script needs.

[mojy3995]

I believe @SorooshMani-NOAA has the same issue #49 .

[SorooshMani-NOAA]

Thanks @mojy3995,

I think I would categorize it into two separate but related issues. The main reason behind both is that pyschism needs internet, and compute nodes on HPCs (as opposed to head nodes) usually don't provide internet access.

• In case of #49 the issue is that param module requires SCHSIM template, but it always tries to download it at https://github.com/schism-dev/pyschism/blob/96e52fd54ff9beacc5e30e7fe2821989d0b67bd4/pyschism/param/schism_init.py#L148-L157 this results in an error while just importing pyschism in Python on a compute node, making it impossible to do anything with it at all for larger setup domains.
• The second issue, such as the current one (#51) is about the fact that, for certain forcings, internet connection is needed. The requirement for this case is understandable, however still it makes a lot of sense to provide an easy way of passing in the cached data to avoid calls to internet services. @cuill can you please remind us where the cached data for NWM should reside if @mojy3995 wants to manually download them from the head node before proceeding to process them on the compute node?
  • On a side note, since pyschism already has all the know how of downloading certain types of data, it will be very useful to provide functions/methods that only download the data without processing them, so that the data gathering and processing stages are separated.
  • It seems that *Inventory* classes in forcing/source_sink/nwm.py are the way to go for only downloading the data. @cuill do you suggest using these classes directly or is there a better way to just download the data?

[mojy3995]

@SorooshMani-NOAA, I am not using NWM results. The script that I am using from Linlin, merges precipitation data from sflux with the river discharges. Both river discharge files and sflux were downloaded on my directory. So there is no connection issue for downloading data.

[SorooshMani-NOAA]

I didn't realize that you already have the data. Then the issues in both tickets are probably the same thing.

[mojy3995]

Thank you, Linlin. It works now.