search

CSHS-CWRA / RavenPy

A Python wrapper to setup and run the hydrologic modelling framework Raven
https://ravenpy.readthedocs.io
MIT License
25 stars 5 forks source link

Kernel dying when repeatedly accessing OutputReader.hydrograph #288

Open huard opened 1 year ago

huard commented 1 year ago

Description

The CANOPEX notebook had kernel failures that seem to be resolved by accessing out.hydrograph.q_sim only once. For some reason, making multiple calls to xr.open_dataset seem to create problems.

This probably calls for caching the opened file.

huard commented 11 months ago

Similar issue observed with #320, where repeated calls to from_nc methods to create the model configuration slowed down the calculations and generated apparently random kernel failures.

Zeitsperre commented 11 months ago

This is definitely a cause of many failures, both here and elsewhere. The tests that tend to fail with segfaults are those that rely on many open_dataset calls. Whether this is related to the async troubles with libnetcdf or not, we should try reducing the read operations if possible. Maybe having pytest fixtures that yield copied objects would be safer?

huard commented 11 months ago

This issue is seen in notebooks that describe real use cases, not in the test suite, so it's bit trickier. I think for the moment we can "solve" this by documenting the problem. Could you link to online resources that capture the issue?