Grid threading - Githubissues

USDA-ARS-NWRC / smrf

SMRF was designed to increase the flexibility of taking measured weather data, or atmospheric models, and distributing the data across a watershed.

Other

12 stars 4 forks source link

Grid threading #118

Closed scotthavens closed 5 years ago

scotthavens commented 5 years ago

There was a bit going on for why the threading didn't work with grid:

the thermal cloud factors between the two different types of gridded methods
HRRR loading loads extra data which needed to be trimmed
The full SMRF test wasn't properly cleaning up after completion, had to add a tearDown
thread_variables was on the SMRF class and needed to be moved to the instance
Added lots of debugging logs so hopefully in the future it's easier to figure out what might be happening

Added a test which works with the WRF, HRRR and generic NetCDF files. This fixes issue #114

scotthavens commented 5 years ago

Upon a longer run, SMRF is still hanging at random points. There has been adequate logging enabled to show that this is most likely not a queue problem. The only other thing that I can think it could be is a subprocess command that is not exiting properly but not terminating the thread. So this becomes a little bigger issue to remove some of the calls to IPW functions throughout SMRF. There are a few that are targeted at the radiation processes so it will take some time to convert them over. PR #122 is an example of converting the sunang calculation from C to Python.

For now, I'm closing the PR as it's not fully working.

micahjohnson150 commented 5 years ago

Dang. So you think its the IPW calls? Watch out stoporad...we coming for you...