search

wpeterman / ResistanceGA

Optimize resistance surfaces using Genetic Algorithms
36 stars 15 forks source link

Analysis stuck without progress #39

Closed pdchnsky closed 7 months ago

pdchnsky commented 1 year ago

Hello, I'm running a multi-surface analysis using the MS_optim function in a script on a computing cluster.

GA.inputs <- GA.prep(ASCII.dir = fullmod, Results.dir = "Results/")
jl.inputs <- jl.prep(n.Pops = length(sp.dat), response = lower(gen.dist), CS_Point.File = sp.dat, JULIA_HOME = "/cm/shared/public/apps/julia/1.8.5/bin$
jl.optim <- MS_optim(jl.inputs = jl.inputs, GA.inputs = GA.inputs)

I'm running these program versions: 

R/4.2.2
julia/1.8.5
proj/6.3.1
geos/3.11.1
gdal/3.6.1-gcc9

When I run the analysis it starts without a problem and runs for over a week. When the analysis finishes, it only outputs an .rds file with the GA results. I can look at the standard error output and it appears to run and iterate okay until it starts to repeat reading maps and doesn't seem to be communicating with Circuitscape anymore.

Screen Shot 2023-09-12 at 2 30 46 PM

I was running this again after a successful first run. I didn't change the script. Do you have any ideas what might be causing this issue? Thank you!

wpeterman commented 1 year ago

This is the first time I've encountered this issue. Is the .rds file complete so that you can extract your results?

On Tue, Sep 12, 2023 at 3:56 PM pdchnsky @.***> wrote:

Hello, I'm running a multi-surface analysis using the MS_optim function in a script on a computing cluster.

GA.inputs <- GA.prep(ASCII.dir = fullmod, Results.dir = "Results/") jl.inputs <- jl.prep(n.Pops = length(sp.dat), response = lower(gen.dist), CS_Point.File = sp.dat, JULIA_HOME = "/cm/shared/public/apps/julia/1.8.5/bin$ jl.optim <- MS_optim(jl.inputs = jl.inputs, GA.inputs = GA.inputs)

I'm running these program versions:

R/4.2.2 julia/1.8.5 proj/6.3.1 geos/3.11.1 gdal/3.6.1-gcc9

When I run the analysis it starts without a problem and runs for over a week. When the analysis finishes, it only outputs an .rds file with the GA results. I can look at the standard error output and it appears to run and iterate okay until it starts to repeat reading maps and doesn't seem to be communicating with Circuitscape anymore. [image: Screen Shot 2023-09-12 at 2 30 46 PM] https://user-images.githubusercontent.com/89862962/267444495-0b0ae415-b001-4b71-9791-dd9940521bf4.png

I was running this again after a successful first run. I didn't change the script. Do you have any ideas what might be causing this issue? Thank you!

— Reply to this email directly, view it on GitHub https://github.com/wpeterman/ResistanceGA/issues/39, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDLQUI73HKJHORX6XREZ7DX2C5ALANCNFSM6AAAAAA4VK7G5I . You are receiving this because you are subscribed to this thread.Message ID: @.***>

pdchnsky commented 1 year ago

The regular output for a successful run includes the files below. There is a diagnostic plots .tiff in the plot folder. The full.rds and .rds files are the same and contain the GA results.

Screen Shot 2023-09-13 at 12 33 35 PM

I'm not sure about the file completeness, these are the only .rds outputs I have gotten for any of the analyses. I have attached screenshots of the .rds file structure from the most recent run.

Screen Shot 2023-09-13 at 1 03 47 PM Screen Shot 2023-09-13 at 1 05 12 PM Screen Shot 2023-09-13 at 1 05 32 PM
wpeterman commented 1 year ago

Not sure why the .rds file is being written, but no other results files. The @solution slot has the optimized parameter values. There aren't readily accessible functions to recreate all the outputs that ResistanceGA produces.

pdchnsky commented 1 year ago

What should the full.rds files contain? It looks like the full.rds file and the .rds file that I'm getting are the same.

wpeterman commented 1 year ago

That is the full .rds. It contains the results of the GA optimization

On Tue, Sep 26, 2023 at 2:31 PM pdchnsky @.***> wrote:

What should the full.rds files contain? It looks like the full.rds file and the .rds file that I'm getting are the same.

— Reply to this email directly, view it on GitHub https://github.com/wpeterman/ResistanceGA/issues/39#issuecomment-1736070985, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABDLQUKRXPTYYXDCNLTVENDX4MNRBANCNFSM6AAAAAA4VK7G5I . You are receiving this because you commented.Message ID: @.***>

pdchnsky commented 7 months ago

Hello, This issue was resolved. It turns out that R on the computing cluster was assigning a default temp directory and that was running out of space. ResistanceGA was attempting to write to the temp file but couldn't and then was searching for those files which weren't there. Assigning a user-defined temp file location through R fixed this issue. Thank you!