Reorganize documentation to become more modeller oriented

[gijsber]

What:

• Reorganize the home page to become a better entry point for modellers.
• Add a Concepts page to provide a more elaborate explanation of the concepts.

Why the current documentation has a strong developer and technical reference orientation (=good), but lacks a conceptual introduction.

How

• [ ] reorganize home page https://deltares.github.io/Ribasim/ into table of contents:

• introduction

• Conceptualization (physical-control-allocation layer)

• Development planning

• About the code

• Downloads

• [ ] add https://deltares.github.io/Ribasim/concepts page. Table of contents:

• Water balance in time & space

• Control concepts

• Equations

• Allocation

Respective related tasks:

• [ ] move section 3 of https://deltares.github.io/Ribasim/ to Concepts page
• [ ] add descriptive section
• [ ] move water balance equations from https://deltares.github.io/Ribasim/core/equations.html to Concepts page
• [ ] add descriptive explanation including sequence digram of nested allocation
• [ ] move Allocation section from https://deltares.github.io/Ribasim/core/allocation.html to Concepts page

[gijsber]

This is my suggestion for the update of the Home page (https://deltares.github.io/Ribasim/)

Introduction

Decision makers need to balance the supply and demand of water in complex situations at the river basin scale and under increasing pressure from climate change. RIBASIM provides a free software tool to visualize, evaluate and prioritize water allocation strategies based on their impact. This provides insights to decision makers, enabling them to build consensus amongst water users and make smart decisions about how to manage water resources optimally considering uncertainties now and in the future. RIBASIM 9 is a water balance simulator composed of 3 conceptual layers: a physical layer representing water bodies and associated infrastructure, a rule-based control layer to manage the infrastructure, and a priority-based allocation layer to take centralized decisions on user abstractions. Typically hydrological processes on land will be represented by other models which can be coupled (online) to Ribasim 9. Currently, an online coupling with Modflow 6 (groundwater) and with Metaswap+Modflow 6 (unsaturated zone + groundwater) is available. Ribasim 9 is the follow up of Ribasim 7 (applied world wide), and SIMRES (applied in the Netherlands), and the surface water models Distribution Model and Mozart of the Dutch National Hydrologhical Instrument.

Conceptualization

Ribasim is conceptually organized in three layers:

• a physical layer
• a rule-based control layer
• an allocation layer

The physical layer

To represent the physical characteristics of the water system in an area, Ribasim allows you to divide the area into a network of connected representative elementary watersheds (Reggiani, Sivapalan, and Majid Hassanizadeh 1998). Within Ribasim, these elements are called basins, which are essentially buckets or reservoirs holding an aggregated volume of water bodies in an area. Basins are chained in a graph with connector nodes determining the exchange of water between the basins. These connector nodes can represent open water connections (e.g. bifurcations or resistance in a free flowing open water channel) or infrastructure elements such as pumps, gates or weirs.

The control layer

Infrastructure elements are often controlled by humans to implement a certain water management strategy. Ribasim allows the configuratrion of conditional rules to influence the exchange of water between basins, either by controlling the inflow or outflow, or by controlling a water level . Control rules evaluate one or multiple conditions to change a parameter setting of an infrastructure element when the conditional criteria are met. Conditions can be either calculated values within the network as well as boundary conditions or (todo) external observations, i.e. observation values external to the model.

The allocation layer

Ribasim allows water users (water demands) to abstract water from the basins (i.e. from the physical layer) unless the water level drops below a minimum level. Under dry conditions, water managers may want to prioritize some abstractions over other abstractions. The Ribasim allocation layer can take care of this prioritization by reducing the abstraction rates of lower-priority demands to ensure that sufficient water remains available in the system for the higher-priority demands.

The layers and the main components and dataflows between the layers are shown in the next figure:

---
title: Conceptual model
---
flowchart RL

subgraph physical[physical layer]
    user-->|abstraction|basin
    basin<-->|flow|connector[basin connector]
end
subgraph rbc[rule based control layer]
    control[control rules]:::system
end
subgraph allocation[allocation layer]
    alloc[global allocation]:::system
end
user-->|request demand|alloc
alloc-->|assign allocation|user
basin-->|volume|alloc
basin-->|volume or level|control
alloc-->|update setpoint|control
control-->|action|connector

Nested allocation Since water systems may be extensive, like in the Netherlands, Ribasim models may become massive networks with multiple 10,000’s of nodes. To keep a proper functioning allocation concept under these circumstances, the modeller can decompose the network domain into a main network and multiple sub-networks. The allocation will then be conducted in three steps:

1. conduct an inventory of demands from the sub-networks to inlets from the main network;
2. allocate the available water in the main network to the subnetworks inlets;
3. allocate the assigned water within each subnetwork to the individual water users.

The users then know what they can extract from the water system in the upcoming period and the rule-based simulation will determine the actual abstraction.

The following sequence diagram illustrates this calculation process.

---
title: Calculation process with nested allocation
---

sequenceDiagram

participant boundary
participant basin
participant user
participant allocation_subNetwork
participant allocation_mainNetwork

user->>allocation_subNetwork: demand
par
   allocation_subNetwork-->>allocation_mainNetwork: demand inventory at inlets
end
boundary->>allocation_mainNetwork: source availablility
basin->>allocation_mainNetwork: source availablility
allocation_mainNetwork-->>allocation_mainNetwork: allocate to inlets
allocation_mainNetwork->>allocation_subNetwork: allocated
par
   allocation_subNetwork-->>allocation_subNetwork: allocate to users
end
allocation_subNetwork->>user: allocated
user->>basin: abstracted

Development planning

Remember: reality may deviate from this planning

• Online coupling with Modflow-Metaswap – 2024 Q1
• Offline coupling with Delwaq - 2024 Q1
• Online coupling with Delwaq – 2024 Q2
• Release ready for project application – Summer 2024
• Off line coupling with wflow – 2023 Q3
• Online coupling with wflow – 2024 Q4

About the code

The kernel of Ribasim is written in the Julia programming language and is built on top of the SciML: Open Source Software for Scientific Machine Learning libraries, notably DifferentialEquations.jl. Ribasim has two API’s, both written in the Python programming language. The Ribasim python package (https://deltares.github.io/Ribasim/python/) offers an API to build, update and analyze Ribasim models programmatically. For runtime data exchange and coupling with other models, ribasim_api is utilized which implements the Basic Modelling Interface BMI (https://bmi-spec.readthedocs.io/en/latest/).

The Ribasim QGIS plugin allows users to construct a model from scratch without programming. For specific tasks, like adding observed rainfall timeseries, it can be faster to use Python instead.

One can also use Ribasim Python to build entire models from base data, such that your model setup is fully reproducible.

See Usage for more information.

%%| file: assets/c4_system.mmd
%%| fig-cap: "System overview of Ribasim"

Downloads

• Ribasim executable - Linux: ribasim_cli_linux.zip
• Ribasim executable - Windows: ribasim_cli_windows.zip
• QGIS plugin: ribasim_qgis.zip.
• Generated testmodels: generated_testmodels.zip

The nightly builds contain the latest developments and can be found below. It is important to either use the release or nightly for all components.

• Ribasim executable: ribasim_cli.zip.
• QGIS plugin: ribasim_qgis.zip.

The Ribasim python package is registered in PyPI and can therefore be installed with pip:

pip install ribasim

For wheel (.whl) downloads, including nightly builds, see the download section. After downloading wheels can be installed by referring to the correct path:

pip install path/to/ribasim-*.whl

[gijsber]

Parking lot for the sequence diagram explaining the nested allocation and interaction with the main loop

sequenceDiagram

participant boundary
participant basin
participant user
participant allocation_subNetwork
participant allocation_mainNetwork

user->>allocation_subNetwork: demand
par
   allocation_subNetwork-->>allocation_mainNetwork: demand inventory at inlets
end
boundary->>allocation_mainNetwork: source availablility
basin->>allocation_mainNetwork: source availablility
allocation_mainNetwork-->>allocation_mainNetwork: allocate to inlets
allocation_mainNetwork->>allocation_subNetwork: allocated
par
   allocation_subNetwork-->>allocation_subNetwork: allocate to users
end
allocation_subNetwork->>user: allocated
user->>basin: abstracted

[visr]

Parking lot

Just to be clear, make sure you keep the mermaid syntax. In Quarto we use this rather than a PNG render, see https://github.com/Deltares/Ribasim/blob/605e5b9b20eea77cd29ab3f39f06f7bfe6933571/docs/core/index.qmd#L15-L39.

[gijsber]

No worries, if you go to edit mode in the comment you get the mermaid syntax for both figures. Github does the rendering to a picture

[gijsber]

Conceptual differences with other model codes

Ribasim 9 versus Ribasim 7 Ribasim 7 is a heuristic water allocation model where prioritized water demands are met from preferential sources such as inflows or surface water reservoirs (i.e. dams). Nodes in the network typically do not have a storage component (except for reservoir nodes or natural lakes). typical timesteps therefore are 10-day, by-monthly or monthly as this allows a drop of water to reach the end of the network within the timestep. Ribasim 7 allows flow direction in one-way only (ie downstream), while demands can be propagated upstream to reservoirs or diversions. Ribasim 7 allows various formulations to compute water demand for agriculture or plublic water supply, while it also offers a reservoir node with hydropower oriented operations. The water distribution is primarily derived by matching source availability with demands given reservoir rules (flood control-target and firm levels) and hedging rules) local flow-based control options for diversions and bifurcations. Ribasim 9 deviates from this paradigm as its water balance is not driven by a demand-allocation matching algorith. Instead it is driven by a numerical scheme that solves the water balance equation for a chain of basins (buckets) with storages which are connected by elements that determine the flow exchange between the basins based on Qh-relations, resistance (i.e. a head difference) or control rules. This numerical scheme chooses its internal timestepping. Compared to Ribasim 7, Ribasim 9 allows much more flexibility to define control rules. Ribasim 9 also deviates on the allocation paradigm as its allocation algorithm primarily is focussed on deciding where water abstractions need to be reduced - in the rule-based simulation - to keep sufficient water in the system for high-prioirity users. Also in terms of water demand, Ribasim 9 deviates from Ribasim 7, as the kernel is given a user/abstraction demand, either via a preprocessed timeseries, or via an online coupling with other models (e.g. unsaturated zone or groundwater models). Ribasim 9 versus D-Hydro

[SouthEndMusic]

This is my suggestion for the update of the Home page (https://deltares.github.io/Ribasim/)

I like it, I do have some comments:

Notes:

• 'either by controlling the inflow or outflow, or by controlling a water level'. We do support level control by PID control or discrete control, but we do not support flow control (other than setting flow rates directly by aforementioned control mechanisms). Trying to control flow lead to some nasty bugs and conceptual problems.
• Regarding the first figure of the allocation layer: updating a setpoint is an interesting one. It is not especially dificult to do, but I do want to point out that this is conceptually different from what control has done upto this point, because updating a setpoint of discrete control changes the control rules themselves. We have to think about validation rules for this.
• 'and the rule-based simulation will determine the actual abstraction.' I think this is conceptually easiest to understand, but:
  1. Whether a user can abstract depending on the level of the abstraction basin is handled with reduction factors in the physical layer
  2. Basin target levels are yet to be properly implemented in allocation (https://github.com/Deltares/Ribasim/issues/564)
• Regarding the second figure:
  1. As discussed with @visr, users are also allowed to be part of the main network directly. It is a matter of what we think is clearer whether we include this in the figures or whether it is mentioned in text only.
  2. In case 'par' stands for parallel: in theory these processes can be done in parallel (if JuMP/Highs indeed also support this), but in practice this is not yet done. We should have a technical discussion with @visr and/or @Hofer-Julian about what is required to support this, and whether for instance the Julia tests can still be run in parallel if the tests themselves run parallel processes.