wpeterman / ResistanceGA

Optimize resistance surfaces using Genetic Algorithms
36 stars 15 forks source link

ResistanceGA can not work with Circuitscape 4.0.5 in Ubuntu 16.04 #8

Closed Clarkhsiung closed 4 years ago

Clarkhsiung commented 5 years ago

I tried to use ResistanceGA in Ubuntu 16.04. I installed Circuitscape 4.0.5 through source. And use this command below.

library(raster)
library(ResistanceGA)
library(gdistance)
library(parallel)
library(doParallel)
data(resistance_surfaces)
dir.create("test_cir")
setwd("~/test_cir/")
write.dir="./"

continuous <- resistance_surfaces[[2]]
writeRaster(continuous,filename =paste0(write.dir,"cont.asc"),overwrite = TRUE)
data(samples)
write.table(samples,file=paste0(write.dir,"samples.txt"),sep="\t",col.names=F,row.names=F)
sample.locales <-SpatialPoints(samples[,c(2,3)])

GA.inputs <-GA.prep(ASCII.dir = write.dir,Results.dir = write.dir,
                    max.cat = 500,
                    max.cont = 500,
                    select.trans = "M",
                    method = "LL",
                    seed = 555)

CS.inputs <- CS.prep(n.Pops = length(samples),
                     CS_Point.File = paste0(write.dir,"samples.txt"),
                     CS.program ='csrun.py',platform="other")

SS_RESULTS <- SS_optim(CS.inputs=CS.inputs,
                       GA.inputs=GA.inputs)

And I get some error message bellow.

Iteration took 0.01 seconds 
    LL = -99999
EXCLUDED TRANSFORMATION

Reading maps
Processing maps
Resistance/conductance map has 2500 nodes
Traceback (most recent call last):
  File "/usr/local/bin/csrun.py", line 13, in <module>
    resistances = cs.compute()
  File "/usr/local/lib/python2.7/dist-packages/circuitscape/profiler.py", line 144, in wrapper
    return func(*args)
  File "/usr/local/lib/python2.7/dist-packages/circuitscape/compute.py", line 31, in compute
    return self.compute_raster()
  File "/usr/local/lib/python2.7/dist-packages/circuitscape/profiler.py", line 144, in wrapper
    return func(*args)
  File "/usr/local/lib/python2.7/dist-packages/circuitscape/compute.py", line 152, in compute_raster
    resistances, solver_failed = self.pairwise_module(self.state.g_map, self.state.poly_map, self.state.points_rc)
  File "/usr/local/lib/python2.7/dist-packages/circuitscape/profiler.py", line 144, in wrapper
    return func(*args)
  File "/usr/local/lib/python2.7/dist-packages/circuitscape/compute.py", line 367, in pairwise_module
    g_habitat = HabitatGraph(g_map=g_map, poly_map=poly_map, connect_using_avg_resistances=self.options.connect_using_avg_resistances, connect_four_neighbors_only=self.options.connect_four_neighbors_only)
  File "/usr/local/lib/python2.7/dist-packages/circuitscape/compute_base.py", line 510, in __init__
    if None != g_map:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Error in file(file, "rt") : cannot open the connection
In addition: Warning messages:
1: In system(paste(CS.inputs$CS.program, CS.ini), hidden) :
  running command 'csrun.py ./tmp/resist_surface.ini' had status 1
2: In file(file, "rt") :
  cannot open file './tmp/resist_surface_resistances.out': No such file or directory

Is it possible to run ResistanceGA with Circuitscape 4.0.5 in Ubuntu 16.04?

wpeterman commented 5 years ago

The development version of ResistanceGA supports use of the Julia language version of CIRCUITSCAPE. devtools::install_github("wpeterman/ResistanceGA", ref = "julia_dev")

You'll need to install Julia https://julialang.org/ (>= v 1.0), and then get the Circuitscape package for Julia https://github.com/Circuitscape/Circuitscape.jl.

Use of ResistanceGA isn't too different when using Julia. You'll need to use the jl.prep function prior to running analyses, and this has a few different arguments to specify (e.g., where is Julia located on your system).

Let me know if you have any questions, and good luck!

Bill Peterman Assistant Professor School of Environment and Natural Resources The Ohio State University 2021 Coffey Road Columbus, OH 43210-1085 Phone: 614.292.9795 Web Page https://goo.gl/4Gc15W Terrestrial Wildlife Ecology Lab http://www.twel.osu.edu

Bill Peterman

On Tue, Sep 17, 2019 at 4:35 AM Clarkhsiung notifications@github.com wrote:

I tried to use ResistanceGA in Ubuntu 16.04. I installed Circuitscape 4.0.5 through source https://github.com/Circuitscape/Circuitscape. And use this command below.

library(raster) library(ResistanceGA) library(gdistance) library(parallel) library(doParallel) data(resistance_surfaces) dir.create("test_cir") setwd("~/test_cir/") write.dir="./"

continuous <- resistance_surfaces[[2]] writeRaster(continuous,filename =paste0(write.dir,"cont.asc"),overwrite = TRUE) data(samples) write.table(samples,file=paste0(write.dir,"samples.txt"),sep="\t",col.names=F,row.names=F) sample.locales <-SpatialPoints(samples[,c(2,3)])

GA.inputs <-GA.prep(ASCII.dir = write.dir,Results.dir = write.dir, max.cat = 500, max.cont = 500, select.trans = "M", method = "LL", seed = 555)

CS.inputs <- CS.prep(n.Pops = length(samples), CS_Point.File = paste0(write.dir,"samples.txt"), CS.program ='csrun.py',platform="other")

SS_RESULTS <- SS_optim(CS.inputs=CS.inputs, GA.inputs=GA.inputs)

And I get some error message bellow.

Iteration took 0.01 seconds LL = -99999 EXCLUDED TRANSFORMATION

Reading maps Processing maps Resistance/conductance map has 2500 nodes Traceback (most recent call last): File "/usr/local/bin/csrun.py", line 13, in resistances = cs.compute() File "/usr/local/lib/python2.7/dist-packages/circuitscape/profiler.py", line 144, in wrapper return func(args) File "/usr/local/lib/python2.7/dist-packages/circuitscape/compute.py", line 31, in compute return self.compute_raster() File "/usr/local/lib/python2.7/dist-packages/circuitscape/profiler.py", line 144, in wrapper return func(args) File "/usr/local/lib/python2.7/dist-packages/circuitscape/compute.py", line 152, in compute_raster resistances, solver_failed = self.pairwise_module(self.state.g_map, self.state.poly_map, self.state.points_rc) File "/usr/local/lib/python2.7/dist-packages/circuitscape/profiler.py", line 144, in wrapper return func(*args) File "/usr/local/lib/python2.7/dist-packages/circuitscape/compute.py", line 367, in pairwise_module g_habitat = HabitatGraph(g_map=g_map, poly_map=poly_map, connect_using_avg_resistances=self.options.connect_using_avg_resistances, connect_four_neighbors_only=self.options.connect_four_neighbors_only) File "/usr/local/lib/python2.7/dist-packages/circuitscape/compute_base.py", line 510, in init if None != g_map: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() Error in file(file, "rt") : cannot open the connection In addition: Warning messages: 1: In system(paste(CS.inputs$CS.program, CS.ini), hidden) : running command 'csrun.py ./tmp/resist_surface.ini' had status 1 2: In file(file, "rt") : cannot open file './tmp/resist_surface_resistances.out': No such file or directory

Is it possible to run ResistanceGA with Circuitscape 4.0.5 in Ubuntu 16.04?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/wpeterman/ResistanceGA/issues/8?email_source=notifications&email_token=ABDLQUJ7R2VFB6HTGFZGKSLQKCJDJA5CNFSM4IXMSU72YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HLZFXLQ, or mute the thread https://github.com/notifications/unsubscribe-auth/ABDLQUO3YXB5I6B4X52APG3QKCJDJANCNFSM4IXMSU7Q .

Clarkhsiung commented 5 years ago

Thanks for your reply. I work ResistanceGA 4.1.0.35 with julia 1.2.0. It worked smoothly until SS_optim Here is code I used.

library(ResistanceGA)
continuous <- resistance_surfaces[[2]]
writeRaster(continuous,filename =paste0(write.dir,"cont.asc"),overwrite = TRUE)
data(samples)
write.table(samples,file=paste0(write.dir,"samples.txt"),sep="\t",col.names=F,row.names=F)
# Create a spatial points object for plotting
sample.locales <-SpatialPoints(samples[,c(2,3)])

GA.inputs <-GA.prep(ASCII.dir = write.dir,Results.dir=write.dir,
                    max.cat = 500,
                    max.cont = 500,
                    select.trans = "M",
                    method = "LL",
                    seed = 555)

CS.inputs <- jl.prep(n.Pops = length(samples),
                     CS_Point.File = paste0(write.dir,"samples.txt"), 
                     parallel = T,core=4,JULIA_HOME = "/home/liao/software/julia-1.2.0/bin/")

SS_RESULTS <- SS_optim(jl.inputs=CS.inputs,GA.inputs=GA.inputs)

I get some error message when I process SS_RESULTS <- SS_optim(jl.inputs=CS.inputs,GA.inputs=GA.inputs)

GA | iter = 25 | Mean = -99999 | Best = -99999
Error: Error happens in Julia.
SystemError: opening file "/home/liao/.julia/environments/v1.2/Project.toml": Too many open files
Stacktrace:
 [1] #systemerror#44(::Nothing, ::typeof(systemerror), ::String, ::Bool) at ./error.jl:134
 [2] systemerror at ./error.jl:134 [inlined]
 [3] #open#311(::Nothing, ::Nothing, ::Nothing, ::Nothing, ::Nothing, ::typeof(open), ::String) at ./iostream.jl:289
 [4] open at ./iostream.jl:281 [inlined]
 [5] #open#312(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{,Tuple{}}}, ::typeof(open), ::getfield(Base, Symbol("##656#657")){String,String}, ::String) at ./iostream.jl:373
 [6] open at ./iostream.jl:373 [inlined]
 [7] explicit_project_deps_get at ./loading.jl:489 [inlined]
 [8] project_deps_get(::String, ::String) at ./loading.jl:305
 [9] identify_package(::String) at ./loading.jl:220
 [10] identify_package(::Base.PkgId, ::String) at ./loading.jl:206
 [11] identify_package at ./loading.jl:200 [inlined]
 [12] require(::Module, ::Symbol) at ./loa

Is there any code/package/software I need to adjust?

wpeterman commented 5 years ago

Not certain about the issue you're having, but try the following:

#################

Create a spatial points object for plotting

sample.locales <-SpatialPoints(samples[,c(2,3)])

GA.inputs <-GA.prep(ASCII.dir = write.dir,Results.dir=write.dir, max.cat = 500, max.cont = 500, select.trans = "M", method = "LL", seed = 555)

CS.inputs <- jl.prep(n.Pops = length(samples), CS_Point.File = sample.locales, parallel = T,core=4,JULIA_HOME = "/home/liao/software/julia-1.2.0/bin/")

SS_RESULTS <- SS_optim(jl.inputs=CS.inputs,GA.inputs=GA.inputs)

#################

The function to run Julia with ResistanceGA has been updated to accept a SpatialPoints object directly.

Bill Peterman Assistant Professor School of Environment and Natural Resources The Ohio State University 2021 Coffey Road Columbus, OH 43210-1085 Phone: 614.292.9795 Web Page https://goo.gl/4Gc15W Terrestrial Wildlife Ecology Lab http://www.twel.osu.edu

Bill Peterman

On Wed, Sep 18, 2019 at 4:49 AM Clarkhsiung notifications@github.com wrote:

Thanks for your reply. I work ResistanceGA 4.1.0.35 with julia 1.2.0. It worked smoothly until SS_optim Here is code I used.

library(ResistanceGA) continuous <- resistance_surfaces[[2]] writeRaster(continuous,filename =paste0(write.dir,"cont.asc"),overwrite = TRUE) data(samples) write.table(samples,file=paste0(write.dir,"samples.txt"),sep="\t",col.names=F,row.names=F)

Create a spatial points object for plotting

sample.locales <-SpatialPoints(samples[,c(2,3)])

GA.inputs <-GA.prep(ASCII.dir = write.dir,Results.dir=write.dir, max.cat = 500, max.cont = 500, select.trans = "M", method = "LL", seed = 555)

CS.inputs <- jl.prep(n.Pops = length(samples), CS_Point.File = paste0(write.dir,"samples.txt"), parallel = T,core=4,JULIA_HOME = "/home/liao/software/julia-1.2.0/bin/")

SS_RESULTS <- SS_optim(jl.inputs=CS.inputs,GA.inputs=GA.inputs)

I get some error message when I process SS_RESULTS <- SS_optim(jl.inputs=CS.inputs,GA.inputs=GA.inputs)

GA | iter = 25 | Mean = -99999 | Best = -99999 Error: Error happens in Julia. SystemError: opening file "/home/liao/.julia/environments/v1.2/Project.toml": Too many open files Stacktrace: [1] #systemerror#44(::Nothing, ::typeof(systemerror), ::String, ::Bool) at ./error.jl:134 [2] systemerror at ./error.jl:134 [inlined] [3] #open#311(::Nothing, ::Nothing, ::Nothing, ::Nothing, ::Nothing, ::typeof(open), ::String) at ./iostream.jl:289 [4] open at ./iostream.jl:281 [inlined] [5] #open#312(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{,Tuple{}}}, ::typeof(open), ::getfield(Base, Symbol("##656#657")){String,String}, ::String) at ./iostream.jl:373 [6] open at ./iostream.jl:373 [inlined] [7] explicit_project_deps_get at ./loading.jl:489 [inlined] [8] project_deps_get(::String, ::String) at ./loading.jl:305 [9] identify_package(::String) at ./loading.jl:220 [10] identify_package(::Base.PkgId, ::String) at ./loading.jl:206 [11] identify_package at ./loading.jl:200 [inlined] [12] require(::Module, ::Symbol) at ./loa

Is there any code/package/software I need to adjust?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/wpeterman/ResistanceGA/issues/8?email_source=notifications&email_token=ABDLQULTP6VLL66HMGYEAKTQKHTTBA5CNFSM4IXMSU72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD67KI3Y#issuecomment-532587631, or mute the thread https://github.com/notifications/unsubscribe-auth/ABDLQUNOFPS2F7KF7K5EWDTQKHTTBANCNFSM4IXMSU7Q .

Clarkhsiung commented 5 years ago

Thank for your kindly advice. After that, I tried to reproduce error message what I did, but I can get the same error message. I think that is my computer problem. And I get stable error message as below.

SS_RESULTS =SS_optim(jl.inputs=CS.inputs,GA.inputs=GA.inputs)
        Iteration took 0.01 seconds
        LL = -99999
EXCLUDED TRANSFORMATION

[ Info: 2019-09-21 11:34:39 : Precision used: None
[ Info: 2019-09-21 11:34:39 : Starting up Circuitscape to use 4 processes in parallel
[ Info: 2019-09-21 11:34:48 : Reading maps
[ Info: 2019-09-21 11:34:48 : Resistance/Conductance map has 2500 nodes
[ Info: 2019-09-21 11:34:52 : Solver used: CHOLMOD
[ Info: 2019-09-21 11:34:52 : Graph has 2500 nodes, 25 focal points and 1 connected components
[ Info: 2019-09-21 11:34:52 : Total number of pair solves = 300
[ Info: 2019-09-21 11:34:52 : Triggering resistance calculation shortcut
[ Info: 2019-09-21 11:34:52 : Total number of pair solves has been reduced to 24
[ Info: 2019-09-21 11:34:52 : Time taken to construct cholesky factor = 0.00435135
[ Info: 2019-09-21 11:34:52 : Time taken to construct local nodemap = 7.2695e-5 seconds
[ Info: 2019-09-21 11:34:52 : Solving points 1 to 24
[ Info: 2019-09-21 11:34:57 : Time taken to complete job = 9.85654914
Error in eval(predvars, data, env) : object 'gd' not found

And here is code what I used.

library(ResistanceGA)
write.dir="/home/liao/test_cir/test/"
continuous <- resistance_surfaces[[2]]
writeRaster(continuous,filename =paste0(write.dir,"cont.asc"),overwrite = TRUE)
data(samples)
write.table(samples,file=paste0(write.dir,"samples.txt"),sep="\t",col.names=F,row.names=F)
# Create a spatial points object for plotting
sample.locales <-SpatialPoints(samples[,c(2,3)])
GA.inputs <-GA.prep(ASCII.dir =continuous,Results.dir = write.dir,
                    max.cat = 500,run = 5,
                    max.cont = 500,
                    select.trans = "M",
                    method = "LL",
                    seed = 555)
CS.inputs <- jl.prep(n.Pops = length(samples),run_test = T,
                     CS_Point.File = sample.locales,
                     parallel = T,core=4,JULIA_HOME = "/home/liao/software/julia-1.2.0/bin/")

SS_RESULTS =SS_optim(jl.inputs=CS.inputs,GA.inputs=GA.inputs)

By the way, when I set parallel = TRUEin GA.perp I will get fatal error in Julia. Julia will be zombie process, and can not work in R. image

wpeterman commented 5 years ago

It doesn't look like you've specified a genetic distance. This is necessary to optimize a surface. Try this code:

GA.inputs <-GA.prep(ASCII.dir =continuous,Results.dir = write.dir, max.cat = 500,run = 5, max.cont = 500, select.trans = "M", method = "LL", seed = 555) CS.inputs <- jl.prep(n.Pops = length(samples),run_test = T, response = lower(Dc_list$Dc_cont), CS_Point.File = sample.locales, parallel = T,core=4,JULIA_HOME = "/home/liao/software/julia-1.2.0/bin/")

SS_RESULTS =SS_optim(jl.inputs=CS.inputs,GA.inputs=GA.inputs)

Bill Peterman Assistant Professor School of Environment and Natural Resources The Ohio State University 2021 Coffey Road Columbus, OH 43210-1085 Phone: 614.292.9795 Web Page https://goo.gl/4Gc15W Terrestrial Wildlife Ecology Lab http://www.twel.osu.edu

Bill Peterman

On Fri, Sep 20, 2019 at 11:54 PM Clarkhsiung notifications@github.com wrote:

Thank for your kindly advice. After that, I tried to reproduce error message what I did, but I can get the same error message. I think that is my computer problem. And I get stable error message as below.

SS_RESULTS =SS_optim(jl.inputs=CS.inputs,GA.inputs=GA.inputs) Iteration took 0.01 seconds LL = -99999 EXCLUDED TRANSFORMATION

[ Info: 2019-09-21 11:34:39 : Precision used: None [ Info: 2019-09-21 11:34:39 : Starting up Circuitscape to use 4 processes in parallel [ Info: 2019-09-21 11:34:48 : Reading maps [ Info: 2019-09-21 11:34:48 : Resistance/Conductance map has 2500 nodes [ Info: 2019-09-21 11:34:52 : Solver used: CHOLMOD [ Info: 2019-09-21 11:34:52 : Graph has 2500 nodes, 25 focal points and 1 connected components [ Info: 2019-09-21 11:34:52 : Total number of pair solves = 300 [ Info: 2019-09-21 11:34:52 : Triggering resistance calculation shortcut [ Info: 2019-09-21 11:34:52 : Total number of pair solves has been reduced to 24 [ Info: 2019-09-21 11:34:52 : Time taken to construct cholesky factor = 0.00435135 [ Info: 2019-09-21 11:34:52 : Time taken to construct local nodemap = 7.2695e-5 seconds [ Info: 2019-09-21 11:34:52 : Solving points 1 to 24 [ Info: 2019-09-21 11:34:57 : Time taken to complete job = 9.85654914 Error in eval(predvars, data, env) : object 'gd' not found

And here is code what I used.

library(ResistanceGA) write.dir="/home/liao/test_cir/test/" continuous <- resistance_surfaces[[2]] writeRaster(continuous,filename =paste0(write.dir,"cont.asc"),overwrite = TRUE) data(samples) write.table(samples,file=paste0(write.dir,"samples.txt"),sep="\t",col.names=F,row.names=F)

Create a spatial points object for plotting

sample.locales <-SpatialPoints(samples[,c(2,3)]) GA.inputs <-GA.prep(ASCII.dir =continuous,Results.dir = write.dir, max.cat = 500,run = 5, max.cont = 500, select.trans = "M", method = "LL", seed = 555) CS.inputs <- jl.prep(n.Pops = length(samples),run_test = T, CS_Point.File = sample.locales, parallel = T,core=4,JULIA_HOME = "/home/liao/software/julia-1.2.0/bin/")

SS_RESULTS =SS_optim(jl.inputs=CS.inputs,GA.inputs=GA.inputs)

By the way, when I set parallel = TRUE in GA.perp I will get fatal error in Julia. Julia will be zombie process, and can not work in R. [image: image] https://user-images.githubusercontent.com/42957793/65367506-18f49300-dc65-11e9-9725-8cf15714b9b8.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/wpeterman/ResistanceGA/issues/8?email_source=notifications&email_token=ABDLQUIBQLGECFJOQ5ENDR3QKWLIVA5CNFSM4IXMSU72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7IJWEA#issuecomment-533764880, or mute the thread https://github.com/notifications/unsubscribe-auth/ABDLQUNX4DXPVGPLVOMSOOLQKWLIVANCNFSM4IXMSU7Q .