Deltares / Ribasim

Water resources modeling
https://ribasim.org/
MIT License
39 stars 5 forks source link

Remove jacobian sparsity false positives #1577

Closed SouthEndMusic closed 2 months ago

SouthEndMusic commented 3 months ago

Fixes https://github.com/Deltares/Ribasim/issues/1575 Fixes the low_storage_factor part of https://github.com/Deltares/Ribasim/issues/1576

SouthEndMusic commented 3 months ago

Below the effect of this refactor on the Jacobian sparsity is shown for the (old) AGV model. Note the number of stored entries.

Old:

229×229 SparseArrays.SparseMatrixCSC{Float64, Int64} with 975 stored entries:
⎡⠑⢄⠀⠀⠂⠀⠀⠀⠀⠀⠊⠈⠂⠐⠀⠀⠀⠀⠀⠀⠀⠀⠈⠀⠀⠀⠀⠀⠁⠀⠀⠀⠀⠈⠀⠀⠈⠀⠀⠀⎤
⎢⠀⠀⠱⣦⣀⣀⡀⡄⢤⠄⠀⠀⣀⣠⣤⡂⠔⠴⠔⣠⣀⣀⡀⠤⣀⠄⡄⣀⠠⡀⡐⡆⣀⡄⣀⣐⠐⠀⠀⠀⎥
⎢⠈⠀⠀⢸⣛⣼⡌⠀⠀⠀⠉⠈⠁⠀⠀⠀⠀⠀⠈⠀⠁⢡⠈⠀⠈⠁⠈⠉⠐⠀⠀⠈⠊⠀⠢⠀⢑⠈⠀⠀⎥
⎢⠀⠀⠀⠬⠂⠉⢑⢔⡀⠀⠀⠀⠀⠠⠠⠐⠀⠀⠄⠀⡊⠫⠀⠀⠀⠠⠔⠁⠀⠀⡀⡐⠀⠀⠄⠄⠀⠀⠀⠄⎥
⎢⠀⠀⠀⠗⠀⠀⠀⠈⠻⢆⣀⠀⠀⠀⠆⠈⠀⠃⠁⠀⠀⠀⠀⠆⠀⠀⢆⡀⠀⠀⠀⠁⠀⢂⠐⠀⠀⠆⠀⠀⎥
⎢⡊⠀⠀⠀⡃⠀⠀⠀⠀⠘⢱⣶⠁⠀⠀⠀⠀⠀⠈⠀⠁⠀⢶⠀⠀⠀⠈⢰⠂⠄⠀⠀⠀⠈⢠⠀⠂⡍⡄⠀⎥
⎢⢈⠀⠀⣸⠁⠀⠀⡀⠀⠀⠁⠀⠵⢇⣀⠆⠀⠀⡀⠀⡀⢀⠀⡀⠈⠂⣎⠀⠆⠀⠀⠈⢀⠀⠊⠆⠀⠈⠈⠀⎥
⎢⠀⠀⠠⠻⠀⠀⢀⠂⡈⠁⠀⠀⠠⠜⠱⣦⠱⠲⠁⠄⡀⠐⠀⠐⠁⠔⠁⠀⠆⠄⡌⡉⠌⠄⠐⠂⠀⠠⠤⠐⎥
⎢⠀⠀⢐⡅⠀⠀⠀⠀⠤⠀⠀⠀⠀⠀⢱⡂⠕⢅⠤⠂⠀⠀⡁⠀⠀⠁⠀⠀⠀⠀⠀⠀⠀⠨⠠⠁⠀⠀⠀⠀⎥
⎢⠀⠀⠐⣡⠂⠀⠀⠁⠁⠀⠂⠀⠀⠈⠁⠄⠠⠃⠑⢄⡆⠀⠀⠀⠀⠀⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⢸⠅⣀⡮⡈⠀⠀⠁⠀⠀⢈⢀⠈⠀⠀⠈⠉⠵⢇⠠⠠⠦⢠⡈⠀⠀⠠⠀⠁⠂⠀⠀⡅⠀⠀⠁⠀⎥
⎢⠂⠀⠀⡌⠂⠀⠀⠀⠠⠄⠘⠓⠀⠠⢀⠀⠁⠈⠀⠀⠀⡂⠑⢄⠀⠀⠄⠀⠂⠀⠀⠀⠀⠀⠀⠁⠀⠀⠀⢀⎥
⎢⠀⠀⠀⠜⠆⠀⠀⡀⠀⠀⠀⠀⠢⠀⢁⠄⠄⠀⠀⠀⠈⣃⠀⠀⠑⢄⡄⠀⠀⠀⠀⠠⠀⠀⠄⠀⠀⠀⠐⠀⎥
⎢⠀⠀⠀⢩⡆⠀⠔⠁⠈⠱⢂⣀⠊⠙⠁⠀⠀⠀⠁⠀⠂⠈⠀⠁⠀⠉⠱⢆⠀⡀⢌⠈⠀⠀⠀⠀⠀⠠⠀⠀⎥
⎢⠁⠀⠀⠢⠐⠀⠀⠀⠀⠀⠈⠄⠈⠁⠈⠅⠀⠀⠀⠀⠀⡀⠈⠀⠀⠀⠀⠠⠑⢄⠀⠀⠀⠀⠀⠀⠁⠀⠀⠁⎥
⎢⠀⠀⠰⠬⡀⠀⢀⠨⠄⠀⠀⠀⡀⠀⡆⠩⠀⠀⠀⠀⠄⠀⠀⠀⠀⡀⡂⠑⠀⠀⠑⢄⠀⠄⠀⡄⠀⠀⢀⠀⎥
⎢⡀⠀⠀⠼⠊⠀⠀⠀⠠⢀⡀⠀⠀⠐⠂⠅⡀⡀⠀⠀⠈⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠄⠑⢄⠀⡀⠀⠀⠀⠀⎥
⎢⠀⠀⢀⢸⠈⠂⠀⠅⠐⠀⠀⠒⠪⠄⠰⠀⠄⠂⠀⠀⠄⠤⠄⠀⠀⠁⠀⠀⠀⠀⠀⠤⠀⠠⡑⢌⠂⠀⠂⠀⎥
⎢⠂⠀⠐⠀⡑⠐⠀⠀⠠⠄⡌⠤⡀⠀⠀⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡀⠁⠀⠀⠀⠀⠀⠈⠀⡑⢌⠀⠀⎥
⎣⠀⠀⠀⠀⠀⠀⠀⠄⠀⠀⠀⠉⠂⠀⢀⠃⠀⠀⠀⠀⠁⠀⠀⢀⠐⠀⠀⠀⠄⠀⠀⠐⠀⠀⠈⠀⠀⠀⠑⢄⎦

  6.120514 seconds (6.56 M allocations: 3.318 GiB, 2.18% gc time)
Model(ts: 5, t: 2024-01-05T00:00:00)

With the new jacobian_prototype construction:

229×229 SparseArrays.SparseMatrixCSC{Float64, Int64} with 732 stored entries:
⎡⠑⢄⠀⠀⠂⠀⠀⠀⠀⠀⠊⠈⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠀⠀⠀⠀⠀⠁⠀⠀⠀⠀⠀⠀⠀⠈⠀⠀⠀⎤
⎢⠀⠀⠱⣦⣀⡀⠀⡄⠤⠄⠀⠀⠀⢀⢤⡂⠀⠴⠄⢠⡀⣀⡀⠤⢀⠀⡄⢀⠠⠀⡐⡆⡀⠀⠀⣐⠀⠀⠀⠀⎥
⎢⠈⠀⠀⢨⢙⣼⡄⠀⠀⠀⠀⠀⠁⠀⠀⠀⠀⠀⠈⠀⠁⠁⠈⠀⠈⠁⠀⠉⠐⠀⠀⠀⠈⠀⠂⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠬⠂⠈⢑⢄⡀⠀⠀⠀⠀⠀⠠⠐⠀⠀⠄⠀⡊⠃⠀⠀⠀⠀⠔⠁⠀⠀⡀⠐⠀⠀⠀⠄⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠖⠀⠀⠀⠈⠳⢄⣀⠀⠀⠀⠆⠈⠀⠂⠁⠀⠀⠀⠀⠀⠀⠀⢂⡀⠀⠀⠀⠀⠀⢂⠀⠀⠀⠀⠀⠀⎥
⎢⡀⠀⠀⠀⡃⠀⠀⠀⠀⠈⢱⣔⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢦⠀⠀⠀⠀⢐⠂⠄⠀⠀⠀⠈⢀⠀⠀⠀⠀⠀⎥
⎢⢈⠀⠀⣸⠁⠀⠀⡀⠀⠀⠁⠀⠱⢅⣀⠆⠀⠀⠀⠀⡀⢀⠀⠀⠈⠂⣎⠀⠆⠀⠀⠈⢀⠀⠈⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠠⠻⠀⠀⢀⠂⡀⠁⠀⠀⠀⠄⠱⣦⠱⠒⠀⠄⡀⠐⠀⠐⠀⠔⠁⠀⠂⠄⡌⠈⠌⠄⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⢐⡅⠀⠀⠀⠀⠤⠀⠀⠀⠀⠀⢱⡂⠕⢄⠤⠂⠀⠀⠁⠀⠀⠁⠀⠀⠀⠀⠀⠀⠀⠨⠠⠁⠀⠀⠀⠀⎥
⎢⠀⠀⠐⣡⠂⠀⠀⠁⠀⠀⠂⠀⠀⠈⠁⠄⠠⠃⠑⢄⡂⠀⠀⠀⠀⠀⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⢸⠅⣀⡪⡀⠀⠀⠁⠀⠀⢀⢀⠈⠀⠀⠈⠉⠵⢄⠠⠠⠤⢠⡈⠀⠀⠠⠀⠀⠂⠀⠀⡅⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠈⠀⠀⠀⠀⠠⠄⠐⠂⠀⠠⢀⠀⠀⠈⠀⠀⠀⡂⠑⢄⠀⠀⠄⠀⠂⠀⠀⠀⠀⠀⠀⠁⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠜⠄⠀⠀⡀⠀⠀⠀⠀⠢⠀⢁⠀⠀⠀⠀⠀⠈⣁⠀⠀⠑⢄⡀⠀⠀⠀⠀⠀⠀⠀⠄⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⢩⡆⠀⠄⠁⠈⠱⠂⡀⠈⠑⠀⠀⠀⠀⠀⠀⠂⠀⠀⠁⠀⠉⠱⢄⠀⡀⢈⠈⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠁⠀⠀⠢⠐⠀⠀⠀⠀⠀⠈⠄⠀⠁⠈⠅⠀⠀⠀⠀⠀⡀⠈⠀⠀⠀⠀⠠⠑⢄⠀⠀⠀⠀⠀⠀⠁⠀⠀⠀⎥
⎢⠀⠀⠠⠠⡀⠀⢀⠨⠄⠀⠀⠀⠀⠀⡆⠩⠀⠀⠀⠀⠄⠀⠀⠀⠀⡀⡂⠑⠀⠀⠑⢄⠀⠀⠀⡄⠀⠀⠀⠀⎥
⎢⡀⠀⠀⠼⠈⠀⠀⠀⠀⠀⡀⠀⠀⠀⠂⠁⡀⡀⠀⠀⠈⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠄⠑⢄⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⢸⠀⠂⠀⠅⠐⠀⠀⠒⠨⠄⠰⠀⠀⠀⠀⠀⠄⠄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠤⠀⠠⡑⢄⠂⠀⠂⠀⎥
⎢⠂⠀⠐⠀⡑⠐⠀⠀⠠⠄⡌⠤⡀⠀⠀⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡀⠁⠀⠀⠀⠀⠀⠈⠀⡑⠀⠀⠀⎥
⎣⠀⠀⠀⠀⠀⠀⠀⠄⠀⠀⠀⠉⠂⠀⢀⠃⠀⠀⠀⠀⠁⠀⠀⢀⠐⠀⠀⠀⠄⠀⠀⠐⠀⠀⠈⠀⠀⠀⠁⠀⎦

  5.484044 seconds (6.55 M allocations: 3.418 GiB, 2.45% gc time)
Model(ts: 5, t: 2024-01-05T00:00:00)

After replacing low_storage_factor with a VectorContinuousCallback implementation which gets rid of the entries for Pump:

229×229 SparseArrays.SparseMatrixCSC{Float64, Int64} with 563 stored entries:
⎡⠑⢄⠀⠀⠂⠀⠀⠀⠀⠀⠊⠈⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠀⠀⠀⠀⠀⠁⠀⠀⠀⠀⠀⠀⠀⠈⠀⠀⠀⎤
⎢⠀⠀⠱⢤⠀⠀⠀⡄⠤⠄⠀⠀⠀⠀⢠⡂⠀⠐⠀⠠⠀⡀⠀⠠⠀⠀⡄⠀⠠⠀⠀⡄⠀⠀⠀⡀⠀⠀⠀⠀⎥
⎢⠈⠀⠀⢨⢙⣼⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠁⠁⠈⠀⠀⠁⠀⠈⠐⠀⠀⠀⠈⠀⠂⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠬⠂⠈⢑⢄⡀⠀⠀⠀⠀⠀⠠⠀⠀⠀⠀⠀⡊⠁⠀⠀⠀⠀⠔⠀⠀⠀⡀⠀⠀⠀⠀⠄⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠒⠀⠀⠀⠈⠱⢄⣀⠀⠀⠀⠆⠈⠀⠀⠁⠀⠀⠀⠀⠀⠀⠀⢂⡀⠀⠀⠀⠀⠀⢂⠀⠀⠀⠀⠀⠀⎥
⎢⡀⠀⠀⠀⡃⠀⠀⠀⠀⠈⢱⣔⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢦⠀⠀⠀⠀⢐⠂⠄⠀⠀⠀⠈⢀⠀⠀⠀⠀⠀⎥
⎢⢈⠀⠀⣸⠁⠀⠀⡀⠀⠀⠁⠀⠑⢅⣀⠆⠀⠀⠀⠀⡀⢀⠀⠀⠈⠀⣎⠀⠆⠀⠀⠈⢀⠀⠈⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠠⠻⠀⠀⢀⠂⡀⠀⠀⠀⠀⠀⠱⣦⠰⠒⠀⠄⡀⠀⠀⠀⠀⠀⠁⠀⠀⠀⡄⠈⠈⠄⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⢐⡅⠀⠀⠀⠀⠤⠀⠀⠀⠀⠀⢱⡂⠐⢄⠤⠂⠀⠀⠁⠀⠀⠁⠀⠀⠀⠀⠀⠀⠀⠨⠠⠁⠀⠀⠀⠀⎥
⎢⠀⠀⠐⣡⠂⠀⠀⠁⠀⠀⠂⠀⠀⠈⠁⠀⠠⠀⠑⢄⡂⠀⠀⠀⠀⠀⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⢸⠁⣀⡢⡀⠀⠀⠀⠀⠀⠀⢀⠀⠀⠀⠀⠁⠕⢄⠀⠀⠀⢀⡀⠀⠀⠀⠀⠀⠂⠀⠀⡅⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠈⠀⠀⠀⠀⠠⠄⠀⠂⠀⠠⢀⠀⠀⠈⠀⠀⠀⡂⠑⢀⠀⠀⠄⠀⠂⠀⠀⠀⠀⠀⠀⠁⠀⠀⠀⠀⎥
⎢⠀⠀⠀⠜⠄⠀⠀⡀⠀⠀⠀⠀⠠⠀⢁⠀⠀⠀⠀⠀⠈⣁⠀⠀⠐⢄⡀⠀⠀⠀⠀⠀⠀⠀⠄⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⢩⡆⠀⠄⠁⠈⠠⠂⠀⠈⠑⠀⠀⠀⠀⠀⠀⠂⠀⠀⠀⠀⠁⠱⢄⠀⡀⠀⠈⠀⠀⠀⠀⠀⠀⠀⠀⎥
⎢⠁⠀⠀⠢⠐⠀⠀⠀⠀⠀⠈⠄⠀⠁⠈⠅⠀⠀⠀⠀⠀⡀⠈⠀⠀⠀⠀⠠⠑⢄⠀⠀⠀⠀⠀⠀⠁⠀⠀⠀⎥
⎢⠀⠀⠀⠀⡀⠀⢀⠨⠄⠀⠀⠀⠀⠀⡆⠨⠀⠀⠀⠀⠄⠀⠀⠀⠀⡀⡂⠑⠀⠀⠁⢄⠀⠀⠀⡄⠀⠀⠀⠀⎥
⎢⡀⠀⠀⠼⠈⠀⠀⠀⠀⠀⡀⠀⠀⠀⠂⠁⠀⠀⠀⠀⠈⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠄⠑⢄⠀⠀⠀⠀⠀⠀⎥
⎢⠀⠀⠀⢸⠀⠂⠀⠅⠐⠀⠀⠒⠨⠄⠰⠀⠀⠀⠀⠀⠄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠠⠀⠠⡑⠄⠂⠀⠂⠀⎥
⎢⠂⠀⠐⠀⡑⠐⠀⠀⠠⠀⡌⠤⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡀⠁⠀⠀⠀⠀⠀⠈⠀⡑⠀⠀⠀⎥
⎣⠀⠀⠀⠀⠀⠀⠀⠄⠀⠀⠀⠉⠀⠀⢀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠐⠀⠀⠀⠄⠀⠀⠐⠀⠀⠈⠀⠀⠀⠁⠀⎦

  3.979922 seconds (6.55 M allocations: 2.531 GiB, 3.48% gc time)
Model(ts: 5, t: 2024-01-05T00:00:00)
SouthEndMusic commented 2 months ago

This PR will be closed, because: