The reason for this PR was that we were trying to run multiple models, each with a single thread, all sharing a process. The reason for this is some weird memory limit issues running w/ kubernetes on Google Cloud. This is probably an edge case, but we were having issues with all of the chdir calls in runclaw b/c all threads were sharing the same working directory.
I worked on getting rid of the directory switching behavior to meet our purposes. It's possible this is overengineering things, but I think it also helped clean up runclaw in the process. Let me know what you think. I'm happy to keep this on our fork and out of the main clawpack code if you'd rather not mess with the current structure of runclaw
Note that in order for these changes to pass tests, we also need a PR that I'm about to file for geoclaw and amrclaw, since it requires some changes to the write methods for various geoclaw and amrclaw data objects.
A couple notable changes I made:
I implemented the xclawout and xclawerr kwargs that it looks like the previous version of runclaw was ready for (but they were always set to None).
I'm using subprocess.run instead of os.system for a little more control
b/c the setrun.py files in the tests reference topography and fgmax files as relative paths, and because we don't know the name of the tmp directory a priori, I also had to modify the way data objects write their .data files (so that they can reference relative paths in setrun.py and write them out properly as absolute paths). This is why the amrclaw and geoclaw PRs are linked
The reason for this PR was that we were trying to run multiple models, each with a single thread, all sharing a process. The reason for this is some weird memory limit issues running w/ kubernetes on Google Cloud. This is probably an edge case, but we were having issues with all of the
chdir
calls inrunclaw
b/c all threads were sharing the same working directory.I worked on getting rid of the directory switching behavior to meet our purposes. It's possible this is overengineering things, but I think it also helped clean up runclaw in the process. Let me know what you think. I'm happy to keep this on our fork and out of the main clawpack code if you'd rather not mess with the current structure of runclaw
Note that in order for these changes to pass tests, we also need a PR that I'm about to file for geoclaw and amrclaw, since it requires some changes to the write methods for various geoclaw and amrclaw data objects.
A couple notable changes I made:
subprocess.run
instead ofos.system
for a little more controlsetrun.py
files in the tests reference topography and fgmax files as relative paths, and because we don't know the name of the tmp directory a priori, I also had to modify the way data objects write their.data
files (so that they can reference relative paths insetrun.py
and write them out properly as absolute paths). This is why the amrclaw and geoclaw PRs are linked