search

Deltares / Ribasim

Water resources modeling
https://ribasim.org/
MIT License
42 stars 5 forks source link

validation of inputs #24

Closed gijsber closed 1 year ago

gijsber commented 1 year ago

Enable validation of input:

evetion commented 1 year ago

So implement on both levels:

visr commented 1 year ago

This looks interesting for validating Arrow tables: https://github.com/beacon-biosignals/Legolas.jl/. It is used in https://github.com/beacon-biosignals/LegolasFlux.jl/.

evetion commented 1 year ago
gijsber commented 1 year ago

for this moment we can have data struct duplications (julia, plugin, python) if we introduce rules, you want to have it better organized at this moment keep validation in julia ~3d (complexity we may need to touch the structs that affect computation)

For future revisit if we want to move to Rust-CLI (#90) doing validation before handing over to julia. This could be used by python and plugin as well.

evetion commented 1 year ago

Ok, so ideally, we generate a schema (jsonschema) from a single source, and use that in all other code. For me, that looks like:

Julia -> Jsonschema -> Python / QGIS

The main reason is because Julia has to be able to compute with the input, so it has to be compatible there.

One can generate Pydantic BaseModels in Python from jsonschemas. These can be used in Pandera as well, although it might be slower. There's an open issue for Pandera to directly generate Pandera models from jsonschema, feel free to upvote it.

evetion commented 1 year ago