search

Circuitscape / Circuitscape.jl

Algorithms from circuit theory to predict connectivity in heterogeneous landscapes
https://circuitscape.org
MIT License
128 stars 35 forks source link

Julia states my habitat file is empty #366

Closed merijnvdb closed 1 year ago

merijnvdb commented 2 years ago

See below my error, I will also attach the content of my .ini file. My resistance raster is 23MB (and thus not empty)

[Circuitscape Mode] data_type = raster scenario = pairwise

[Version] version = 5.11.2

[Habitat raster or graph] habitat_file = D:\Merijn\Desktop\circuitch2\resraster_new.tif habitat_map_is_resistances = true

[Connection Scheme for raster habitat data] connect_four_neighbors_only = false connect_using_avg_resistances = false

[Short circuit regions (aka polygons)] use_polygons = false polygon_file = False

[Options for advanced mode] ground_file_is_resistances = true source_file = (Browse for a current source file) remove_src_or_gnd = keepall ground_file = (Browse for a ground point file) use_unit_currents = false use_direct_grounds = false

[Mask file] use_mask = false mask_file = None

[Options for one-to-all and all-to-one modes] use_variable_source_strengths = false variable_source_file = None

[Options for pairwise and one-to-all and all-to-one modes] included_pairs_file = (Browse for a file with pairs to include or exclude) use_included_pairs = false point_file = D:\Merijn\Desktop\circuitch2\new_nodes.tif

[Calculation options] solver = cg+amg parallelize = true max_parallel = 3

[Output options] write_cum_cur_map_only = true log_transform_maps = false output_file = D:\Merijn\Desktop\circuitch2\settings write_max_cur_maps = false write_volt_maps = false set_null_currents_to_nodata = false set_null_voltages_to_nodata = false compress_grids = false write_cur_maps = true

julia> compute("D:\Merijn\Desktop\circuitch2\newest.ini") I need to do double backslash to avoid an error: syntax:invalid escape sequence [ Info: 2022-09-19 20:59:00 : Precision used: Double [ Info: 2022-09-19 20:59:01 : Starting up Circuitscape to use 3 processes in parallel [ Info: 2022-09-19 20:59:17 : Reading maps ERROR: "Error: zero resistance values are not currently supported for habitat maps. Use a short-circuit region file instead." Stacktrace: [1] read_cellmap(habitat_file::String, is_res::Bool, #unused#::Type{Float64}) @ Circuitscape C:\Users\merij.julia\packages\Circuitscape\XpftG\src\io.jl:89 [2] load_raster_data(T::Type, V::Type, cfg::Dict{String, String}) @ Circuitscape C:\Users\merij.julia\packages\Circuitscape\XpftG\src\io.jl:429 [3] raster_pairwise(T::Type, V::Type, cfg::Dict{String, String}) @ Circuitscape C:\Users\merij.julia\packages\Circuitscape\XpftG\src\raster\pairwise.jl:18 [4] _compute(T::Type, V::Type, cfg::Dict{String, String}) @ Circuitscape C:\Users\merij.julia\packages\Circuitscape\XpftG\src\run.jl:43 [5] macro expansion @ .\timing.jl:287 [inlined] [6] compute(path::String) @ Circuitscape C:\Users\merij.julia\packages\Circuitscape\XpftG\src\run.jl:31 [7] top-level scope @ REPL[4]:1

merijnvdb commented 2 years ago

In case it helps, here are my raster files (for nodes + habitat map) and .ini file https://we.tl/t-fGP0sIR2N9

ranjanan commented 2 years ago

Your habitat file is not empty, but does contain zeros. At least according to how Circuitscape is reading it: 

x = Circuitscape.read_raster("resraster_new.tif", Float64)
count(iszero, x)

returns

16539

We don't support rasters like that currently because that would result in a conductance matrix with Inf in them. Can you replace the zeros with values that are small relative to the rest of your parametrization?

merijnvdb commented 2 years ago

It still isn't working after changing all zeros in the raster to a different value. I will attach a link to the new files, and please see below the error.

Wetransfer file: https://we.tl/t-Nr0wCHbGZh

ERROR: MethodError: Cannot `convert` an object of type Nothing to an object of type Float32
Closest candidates are:
  convert(::Type{T}, ::T) where T<:Number at number.jl:6
  convert(::Type{T}, ::Number) where T<:Number at number.jl:7
  convert(::Type{T}, ::Base.TwicePrecision) where T<:Number at twiceprecision.jl:250
  ...
Stacktrace:
  [1] (::Circuitscape.var"#70#71"{DataType})(raw::ArchGDAL.Dataset)
    @ Circuitscape C:\Users\mvandenbosch\.julia\packages\Circuitscape\QbA2E\src\io.jl:511
  [2] read(f::Circuitscape.var"#70#71"{DataType}, args::String; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ ArchGDAL C:\Users\mvandenbosch\.julia\packages\ArchGDAL\zkx2f\src\context.jl:267
  [3] read(f::Function, args::String)
    @ ArchGDAL C:\Users\mvandenbosch\.julia\packages\ArchGDAL\zkx2f\src\context.jl:265
  [4] read_raster
    @ C:\Users\mvandenbosch\.julia\packages\Circuitscape\QbA2E\src\io.jl:492 [inlined]
  [5] _grid_reader(T::Type, file::String)
    @ Circuitscape C:\Users\mvandenbosch\.julia\packages\Circuitscape\QbA2E\src\io.jl:106
  [6] read_cellmap(habitat_file::String, is_res::Bool, #unused#::Type{Float64})
    @ Circuitscape C:\Users\mvandenbosch\.julia\packages\Circuitscape\QbA2E\src\io.jl:83
  [7] load_raster_data(T::Type, V::Type, cfg::Dict{String, String})
    @ Circuitscape C:\Users\mvandenbosch\.julia\packages\Circuitscape\QbA2E\src\io.jl:429
  [8] raster_pairwise(T::Type, V::Type, cfg::Dict{String, String})
    @ Circuitscape C:\Users\mvandenbosch\.julia\packages\Circuitscape\QbA2E\src\raster\pairwise.jl:18
  [9] _compute(T::Type, V::Type, cfg::Dict{String, String})
    @ Circuitscape C:\Users\mvandenbosch\.julia\packages\Circuitscape\QbA2E\src\run.jl:43
 [10] macro expansion
    @ .\timing.jl:287 [inlined]
 [11] compute(path::String)
    @ Circuitscape C:\Users\mvandenbosch\.julia\packages\Circuitscape\QbA2E\src\run.jl:31
 [12] top-level scope
    @ REPL[6]:1
ranjanan commented 2 years ago

Looks like you set your nodata value to nothing? Could you set it to something like -9999? The 

julia> f = Circuitscape.ArchGDAL.read("wetransfer_untitled-transfer_2022-10-21_2337\\newraster\\nozeroraster.tif")
GDAL Dataset (Driver: GTiff/GeoTIFF)
File(s):
  wetransfer_untitled-transfer_2022-10-21_2337\newraster\nozeroraster.tif
  wetransfer_untitled-transfer_2022-10-21_2337\newraster\nozeroraster.tif.ovr
  wetransfer_untitled-transfer_2022-10-21_2337\newraster\nozeroraster.tif.aux.xml

Dataset (width x height): 7007 x 3878 (pixels)
Number of raster bands: 1
  [GA_ReadOnly] Band 1 (Gray): 7007 x 3878 (Float32)

julia> band = Circuitscape.ArchGDAL.getband(f, 1)
[GA_ReadOnly] Band 1 (Gray): 7007 x 3878 (Float32)
    blocksize: 128×128, nodata: nothing, units: 1.0px + 0.0
    overviews: (0) 3504x1939 (1) 1752x970 (2) 876x485
               (3) 438x243

In the previous file you set it to something like 

julia> f2 = Circuitscape.ArchGDAL.read("resraster_new.tif")
GDAL Dataset (Driver: GTiff/GeoTIFF)
File(s):
  resraster_new.tif

Dataset (width x height): 7007 x 3878 (pixels)
Number of raster bands: 1
  [GA_ReadOnly] Band 1 (Gray): 7007 x 3878 (Float32)

julia> band2 = Circuitscape.ArchGDAL.getband(f2, 1)
[GA_ReadOnly] Band 1 (Gray): 7007 x 3878 (Float32)
    blocksize: 128×128, nodata: 3.3999999521443642e38, units: 1.0px + 0.0
    overviews:
merijnvdb commented 2 years ago

Hi Ranjan,

I don't really understand how this would work conceptually, if I set NA raster values to a non-zero value, would my NAs (which I don't want to calculate connectivity for) not by definition take a resistance value (the number I assign to it?).

So e.g. I assign 5 to NAs, then my NAs get resistance value 5, which I don't want?

Merijn

Op zo 23 okt. 2022 om 13:06 schreef Ranjan Anantharaman < @.***>:

Looks like you set your nodata value to nothing? Could you set it to something like -9999? The

julia> f = Circuitscape.ArchGDAL.read("wetransfer_untitled-transfer_2022-10-21_2337\newraster\nozeroraster.tif")

GDAL Dataset (Driver: GTiff/GeoTIFF) File(s):

wetransfer_untitled-transfer_2022-10-21_2337\newraster\nozeroraster.tif

wetransfer_untitled-transfer_2022-10-21_2337\newraster\nozeroraster.tif.ovr

wetransfer_untitled-transfer_2022-10-21_2337\newraster\nozeroraster.tif.aux.xml

Dataset (width x height): 7007 x 3878 (pixels)

Number of raster bands: 1

[GA_ReadOnly] Band 1 (Gray): 7007 x 3878 (Float32)

julia> band = Circuitscape.ArchGDAL.getband(f, 1)

[GA_ReadOnly] Band 1 (Gray): 7007 x 3878 (Float32)

blocksize: 128×128, nodata: nothing, units: 1.0px + 0.0

overviews: (0) 3504x1939 (1) 1752x970 (2) 876x485

           (3) 438x243

In the previous file you set it to something like

julia> f2 = Circuitscape.ArchGDAL.read("resraster_new.tif")

GDAL Dataset (Driver: GTiff/GeoTIFF) File(s):

resraster_new.tif

Dataset (width x height): 7007 x 3878 (pixels)

Number of raster bands: 1

[GA_ReadOnly] Band 1 (Gray): 7007 x 3878 (Float32)

julia> band2 = Circuitscape.ArchGDAL.getband(f2, 1)

[GA_ReadOnly] Band 1 (Gray): 7007 x 3878 (Float32)

blocksize: 128×128, nodata: 3.3999999521443642e38, units: 1.0px + 0.0

overviews:

— Reply to this email directly, view it on GitHub https://github.com/Circuitscape/Circuitscape.jl/issues/366#issuecomment-1288155949, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASHJWMNUELL24NMXILWR543WEVWC5ANCNFSM6AAAAAAQQTCBME . You are receiving this because you authored the thread.Message ID: @.***>

merijnvdb commented 2 years ago

Additionally, my study area is a certain shape, but a raster of course is always a standard shape, so everything outside the study area are NA values. These would then also be assigned a certain value and as such included in the connectivity model?

Op zo 23 okt. 2022 om 21:10 schreef Merijn van den Bosch < @.***>:

Hi Ranjan,

I don't really understand how this would work conceptually, if I set NA raster values to a non-zero value, would my NAs (which I don't want to calculate connectivity for) not by definition take a resistance value (the number I assign to it?).

So e.g. I assign 5 to NAs, then my NAs get resistance value 5, which I don't want?

Merijn

Op zo 23 okt. 2022 om 13:06 schreef Ranjan Anantharaman < @.***>:

Looks like you set your nodata value to nothing? Could you set it to something like -9999? The

julia> f = Circuitscape.ArchGDAL.read("wetransfer_untitled-transfer_2022-10-21_2337\newraster\nozeroraster.tif")

GDAL Dataset (Driver: GTiff/GeoTIFF) File(s):

wetransfer_untitled-transfer_2022-10-21_2337\newraster\nozeroraster.tif

wetransfer_untitled-transfer_2022-10-21_2337\newraster\nozeroraster.tif.ovr

wetransfer_untitled-transfer_2022-10-21_2337\newraster\nozeroraster.tif.aux.xml

Dataset (width x height): 7007 x 3878 (pixels)

Number of raster bands: 1

[GA_ReadOnly] Band 1 (Gray): 7007 x 3878 (Float32)

julia> band = Circuitscape.ArchGDAL.getband(f, 1)

[GA_ReadOnly] Band 1 (Gray): 7007 x 3878 (Float32)

blocksize: 128×128, nodata: nothing, units: 1.0px + 0.0

overviews: (0) 3504x1939 (1) 1752x970 (2) 876x485

           (3) 438x243

In the previous file you set it to something like

julia> f2 = Circuitscape.ArchGDAL.read("resraster_new.tif")

GDAL Dataset (Driver: GTiff/GeoTIFF) File(s):

resraster_new.tif

Dataset (width x height): 7007 x 3878 (pixels)

Number of raster bands: 1

[GA_ReadOnly] Band 1 (Gray): 7007 x 3878 (Float32)

julia> band2 = Circuitscape.ArchGDAL.getband(f2, 1)

[GA_ReadOnly] Band 1 (Gray): 7007 x 3878 (Float32)

blocksize: 128×128, nodata: 3.3999999521443642e38, units: 1.0px + 0.0

overviews:

— Reply to this email directly, view it on GitHub https://github.com/Circuitscape/Circuitscape.jl/issues/366#issuecomment-1288155949, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASHJWMNUELL24NMXILWR543WEVWC5ANCNFSM6AAAAAAQQTCBME . You are receiving this because you authored the thread.Message ID: @.***>

ranjanan commented 1 year ago

Closing this due to inactivity, and a subsequent email chain. Please reopen if this is a problem with the latest versions, which have many updates.