search

Circuitscape / Omniscape.jl

Functions to compute omnidirectional landscape connectivity using circuit theory and the Omniscape algorithm.
https://docs.circuitscape.org/Omniscape.jl/stable/
MIT License
58 stars 12 forks source link

Error Message received multiple times: AssertionError: norm(G * v - curr) < if eltype(curr) == Float64 #100

Closed linhunt closed 3 years ago

linhunt commented 3 years ago

I am receiving this error message using different inputs files and on different computers. The run throws this error message but continues to compute on one of the PC's and seems to have frozen on the other PC. This is using Julia 1.6.2 and the newest version of Omniscape. We also successfully ran Omniscape using the same input files but with a smaller block/radius with no error messages. Does this error message suggest that any output we might get from these runs are not trustworthy?

Thanks ![ErrorMessage](https://user-images.githubusercontent.com/40667892/126812612-a365c8f5-f573-4dc8-8361-6f7c9899d8

linhunt commented 3 years ago

I am not able to add the screenshot the error messages are ERROR: TaskFailedException & ERROR: nested task error
they both have long stack trace codes after them which I can try to send to you if that is needed.

vlandau commented 3 years ago

Yes please copy and paste the full stacktrace in a code block if you can!

linhunt commented 3 years ago 
Progress:   1%|█                                                 |  ETA: 11.88 daysERROR: TaskFailedException
Stacktrace:
 [1] wait
   @ .\task.jl:322 [inlined]
 [2] threading_run(func::Function)
   @ Base.Threads .\threadingconstructs.jl:34
 [3] macro expansion
   @ .\threadingconstructs.jl:93 [inlined]
 [4] run_omniscape(cfg::Dict{String, String}, resistance::Matrix{Union{Missing, Float64}}; reclass_table::Matrix{Union{Missing, Float64}}, source_strength::Matrix{Union{Missing, Float64}}, condition1::Matrix{Union{Missing, Float64}}, condition2::Matrix{Union{Missing, Float64}}, condition1_future::Matrix{Union{Missing, Float64}}, condition2_future::Matrix{Union{Missing, Float64}}, wkt::String, geotransform::Vector{Float64}, write_outputs::Bool)
   @ Omniscape C:\Users\rgrundel\.julia\packages\Omniscape\0AlRt\src\main.jl:268
 [5] top-level scope
   @ REPL[6]:1
    nested task error: AssertionError: norm(G * v - curr) < if eltype(curr) == Float64
            TOL_DOUBLE
        else
            TOL_SINGLE
        end
    Stacktrace:
     [1] solve_linear_system(G::SparseArrays.SparseMatrixCSC{Float64, Int64}, curr::Vector{Float64}, M::AlgebraicMultigrid.Preconditioner{AlgebraicMultigrid.MultiLevel{AlgebraicMultigrid.Pinv{Float64}, AlgebraicMultigrid.GaussSeidel{AlgebraicMultigrid.SymmetricSweep}, AlgebraicMultigrid.GaussSeidel{AlgebraicMultigrid.SymmetricSweep}, SparseArrays.SparseMatrixCSC{Float64, Int64}, SparseArrays.SparseMatrixCSC{Float64, Int64}, LinearAlgebra.Adjoint{Float64, SparseArrays.SparseMatrixCSC{Float64, Int64}}, AlgebraicMultigrid.MultiLevelWorkspace{Vector{Float64}, 1}}})
       @ Circuitscape C:\Users\rgrundel\.julia\packages\Circuitscape\Qr6wW\src\core.jl:616
     [2] macro expansion
       @ .\timing.jl:287 [inlined]
     [3] multiple_solve(s::Circuitscape.AMGSolver, matrix::SparseArrays.SparseMatrixCSC{Float64, Int64}, sources::Vector{Float64}, suppress_info::Bool)
       @ Circuitscape C:\Users\rgrundel\.julia\packages\Circuitscape\Qr6wW\src\raster\advanced.jl:311
     [4] multiple_solver(cfg::Dict{String, String}, solver::Circuitscape.AMGSolver, a::SparseArrays.SparseMatrixCSC{Float64, Int64}, sources::Vector{Float64}, grounds::Vector{Float64}, finitegrounds::Vector{Float64})
       @ Circuitscape C:\Users\rgrundel\.julia\packages\Circuitscape\Qr6wW\src\raster\advanced.jl:291
     [5] calculate_current(conductance::Matrix{Union{Missing, Float64}}, source::Matrix{Union{Missing, Float64}}, ground::Matrix{Float64}, cs_flags::Circuitscape.RasterFlags, cs_cfg::Dict{String, String}, T::DataType)
       @ Omniscape C:\Users\rgrundel\.julia\packages\Omniscape\0AlRt\src\utils.jl:410
     [6] solve_target!(i::Int64, n_targets::Int64, int_arguments::Dict{String, Int64}, targets::Matrix{Float64}, source_strength::Matrix{Union{Missing, Float64}}, resistance::Matrix{Union{Missing, Float64}}, os_flags::Omniscape.OmniscapeFlags, cs_cfg::Dict{String, String}, cs_flags::Circuitscape.RasterFlags, o::Circuitscape.OutputFlags, condition1_present::Matrix{Union{Missing, Float64}}, condition1_future::Matrix{Union{Missing, Float64}}, condition2_present::Matrix{Union{Missing, Float64}}, condition2_future::Matrix{Union{Missing, Float64}}, comparison1::String, comparison2::String, condition1_lower::Float64, condition1_upper::Float64, condition2_lower::Float64, condition2_upper::Float64, correction_array::Matrix{Float64}, cum_currmap::Array{Float64, 3}, fp_cum_currmap::Array{Float64, 3}, precision::DataType)
       @ Omniscape C:\Users\rgrundel\.julia\packages\Omniscape\0AlRt\src\utils.jl:497
     [7] macro expansion
       @ C:\Users\rgrundel\.julia\packages\Omniscape\0AlRt\src\main.jl:273 [inlined]
     [8] (::Omniscape.var"#185#threadsfor_fun#15"{Int64, ProgressMeter.Progress, Circuitscape.RasterFlags, Circuitscape.OutputFlags, Int64, Dict{String, String}, Float64, Float64, Float64, Float64, String, String, Omniscape.OmniscapeFlags, DataType, Dict{String, Int64}, UnitRange{Int64}})(onethread::Bool)
       @ Omniscape .\threadingconstructs.jl:81
     [9] (::Omniscape.var"#185#threadsfor_fun#15"{Int64, ProgressMeter.Progress, Circuitscape.RasterFlags, Circuitscape.OutputFlags, Int64, Dict{String, String}, Float64, Float64, Float64, Float64, String, String, Omniscape.OmniscapeFlags, DataType, Dict{String, Int64}, UnitRange{Int64}})()
       @ Omniscape .\threadingconstructs.jl:48
vlandau commented 3 years ago

That appears to be the same problem as #99 which I just reopened. Unfortunately it is an issue with Circuitscape as opposed to Omniscape. I'm going to open an issue in Circuitscape/Circuitscape.jl and link this and the other issue. What is your problem size? (how big are the input rasters?)

vlandau commented 3 years ago

cc @ranjanan

linhunt commented 3 years ago

Oh I'm sorry I didn't realize there was already an issue submitted for this error message. The .tif input file is 1.7GB up until the last few weeks we had been able to run these large problems with no error/warning messages popping up at different block/radius settings. I am currently running one of these larger problems on using Julia 1.5.3 and which ever version of Omniscape was available at the time I installed Julia 1.5.3 (if that makes sense). I hope this additional information may be of use to you. Thank you so much for your assistance.

vlandau commented 3 years ago

No worries! The issue was closed when I should have kept it open. The reason it's not working with the latest version of Omniscape is because that version uses the latest version of Circuitscape, which is where the problem is coming from. I'll open that issue in Circuitscape.jl since that's where this will need to be fixed, and link this issue.

In the mean time, if you force Julia to install Circuitscape 5.7.1: ] to open the Pkg prompt, then rm Circuitscape then add Circuitscape@5.7.1, then install Omniscape 0.4.4 with add Omniscape@0.4.4, the problem shouldn't occur, though you won't have access to the CHOLMOD solver in that case.

vlandau commented 3 years ago

@linhunt can you tell me a bit about your inputs? Are you using just a resistance surface and deriving sources from that, or are you using separate resistance and source strength inputs. Also for any inputs, what is the range of values. I'm just trying to gather some data on the sorts of inputs that seem to trigger this issue to get an idea of where to start to find a solution. Thanks!

linhunt commented 3 years ago

Hi, So the input is a .tif file the cover the Northern Lakes and Forests (basically all of the MI upper peninsula, Northern Wisconsin, Northern Minnesota, and a little further west). The file was created in ArcGIS from the National Landcover Dataset so it is at a 30x30m resolution. I converted the landcover data into resistances in ArcGIS so we aren't using the reclassify_resistance option from the example. We have 3 combinations of four resistance bins or categories, for all of them the minimum value is 1 and the max is 1001 the middle two groups vary between those two ranges ( I can give you those values for each of the three combinations). We have set the sources as pixels with a value of 1. We have two block/radius combinations, block size = 1 radius =10 and block = 101 radius = 1010, our errors seem to only come on runs using the larger block/radius combination. If you would like to see the code I have been using I can add that here, like I said I basically use the same script and just change the file directory, output name, and block/radius. We have completed like 6 of each of the large and small block/radius runs for each of the resistance value combinations without issue. I can also give you information on the computers we are using if it might be an issue with something on the computer side. I hope I have given you enough information, sorry if it is too much.

vlandau commented 3 years ago

No that's all great info thank you! I'll try to see if there are any similarities with inputs from other users that have experienced this issue

ranjanan commented 3 years ago

@linhunt if you could attach your files here I can take a look and debug.

ranjanan commented 3 years ago

https://github.com/JuliaRegistries/General/pull/42367. Release new Omniscape version and test this out?

vlandau commented 3 years ago

@ranjanan yeah I'll release a new patch today!

vlandau commented 3 years ago

This should now be fixed in Circuitscape 5.8.3. Please update Omniscape to it's latest version to ensure the latest version of Circuitscape (which contains the fix) is being used by Omniscape.