Deltares / hydromt

HydroMT: Automated and reproducible model building and analysis
https://deltares.github.io/hydromt/
MIT License
75 stars 30 forks source link

Data catalog file of artifact_data v0.0.9 contains incorrect source names #1087

Open verseve opened 1 month ago

verseve commented 1 month ago

HydroMT version checks

Reproducible Example

The hydromt_wflow tests fail in PR #300.

It seems this is caused by the data_catalog.yml file that is downloaded when building a wflow model during the tests from artifact_data.

Current behaviour

FAILED tests/test_model_class.py::test_model_build[wflow] - FileNotFoundError: No such file found: rivers_lin2019_v1
FAILED tests/test_model_class.py::test_model_build[wflow_sediment] - FileNotFoundError: No such file found: rivers_lin2019_v1
FAILED tests/test_model_class.py::test_model_build[wflow_simple] - FileNotFoundError: No such file found: rivers_lin2019_v1
FAILED tests/test_model_methods.py::test_setup_grid - FileNotFoundError: No such file found: globcover
FAILED tests/test_model_methods.py::test_projected_crs - FileNotFoundError: No such file found: globcover
FAILED tests/test_model_methods.py::test_setup_lai - FileNotFoundError: No such file found: vito
FAILED tests/test_model_methods.py::test_setup_rivers[wflow_dem] - FileNotFoundError: No such file found: rivers_lin2019_v1
FAILED tests/test_model_methods.py::test_setup_rivers[dem_subgrid] - FileNotFoundError: No such file found: rivers_lin2019_v1
FAILED tests/test_model_methods.py::test_setup_floodplains_1d - FileNotFoundError: No such file found: rivers_lin2019_v1
FAILED tests/test_model_methods.py::test_setup_floodplains_2d[wflow_dem] - FileNotFoundError: No such file found: rivers_lin2019_v1
FAILED tests/test_model_methods.py::test_setup_floodplains_2d[dem_subgrid] - FileNotFoundError: No such file found: rivers_lin2019_v1
FAILED tests/test_model_methods.py::test_setup_lulc_sed - FileNotFoundError: No such file found: globcover

Desired behaviour

Source names rivers_lin2019_v1, vito and globcover are not listed in data_catalog.yml. The source names are listed in the data_catalog.yml file from hydromt-artifacts-0.0.9 (Source code). This seems the correct file.

Also, my recommendation would be to store this file at one location...

Additional context

No response

veenstrajelmer commented 1 month ago

Hi @verseve I think this has to do with the update of the artifact data, where some dataset pointers were renamed (e.g. rivers_lin2019_v1 to hydro_rivers_lin2019 and vito to vito_2015). This is the related issue for hydromt_delft3dfm: https://github.com/Deltares/hydromt_delft3dfm/issues/159

It includes some relevant documentations (like the changelog containing old vs new names) and the linked PR shows how we resolved this for hydromt_delft3dfm.

veenstrajelmer commented 1 month ago

@verseve could you confirm this resolved your issue or not?

verseve commented 1 month ago

Hi @veenstrajelmer, yes the tests run fine now( fixed by@hboisgon). The data_catalog.yml in hydromt-artifacts-0.0.9 (Source code) is incorrect. I would recommend to store this file at one location.

veenstrajelmer commented 1 month ago

Ok, if I understand you correctly, you are saying that the data_catalog from hydromt-artifacts and the data_catalog in hydromt are different, even though their versions are identical (0.0.9) right? That indeed seems a bit odd to me. I indeed see vito in the first one and vito_2015 in the latter.

verseve commented 1 month ago

Yes, correct, these two data_catalog.yml files are different, while having identical versions.