search

lkerr / groundfish-MSE

Development of Robust Management Strategies for Northeast Groundfish Fisheries in a Changing Climate
MIT License
4 stars 2 forks source link

Asap dirs cherrypick #272

Open mle2718 opened 1 year ago

mle2718 commented 1 year ago

closes #205

I think I was the one who wrote that odd chunk of code. The issue was that when running in parallel on the HPCC I wanted to do the best I could to not overwrite runs when many iterations of the model were running simultaneously. One approach that is kind of along the lines of @mle2718 's suggestion might be to add a text file counter that gets read/written over every time the folder is queried. I'm thinking that if the count is internalized within an instance of R there might be some overwriting going on. There are surely better ways than random number assignment :) .

This code should work, as long as the runSim.sh script doesn't execute all the runSim.R's at the same time.

jerellejesse commented 1 year ago

Thanks! I'm testing things on the HPCC this week to double check everything!

mle2718 commented 1 year ago

Thanks! I'm testing things on the HPCC this week to double check everything!

I have the beginnings of an idea to deal with this.

mle2718 commented 1 year ago

I added an extra failsafe. I'm not sure if it will work as intended. But basically, if there are two runSims that are started nearly-simultanously, they will check to see if the directory they are trying to make exists.

I'm slightly worried about what happens if 2 different child processes try to use the same ASAP directory. It's probably a very very small concern, because the dir.create() statement will throw an error message if the directory it tries to create already exists.

jerellejesse commented 1 year ago

This looks good and I will double check it soon! The HPCC was giving me some trouble last week and I didn't get to testing yet.