equinor / ert

ERT - Ensemble based Reservoir Tool - is designed for running ensembles of dynamical models such as reservoir models, in order to do sensitivity analysis and data assimilation. ERT supports data assimilation using the Ensemble Smoother (ES), Ensemble Smoother with Multiple Data Assimilation (ES-MDA) and Iterative Ensemble Smoother (IES).
https://ert.readthedocs.io/en/latest/
GNU General Public License v3.0
102 stars 107 forks source link

Validate results after SingleTestRun/EnsembleExperiment #7966

Open oyvindeide opened 4 months ago

oyvindeide commented 4 months ago

Sometimes it is a bit hard for users to tell if all the required observations and parameters are loaded in the callback. In AHM we allow some observations to be missing, and only crash if all observations are deactivated.

The REFCASE has some functionality to check that the correct dates are present for summary data, however there has historically been a problem that it is not necessarily in sync with the schedule file, and as such, might give false positives or negatives.

One way we can improve this, is by validating that SingleTestRun and EnsembleExperiment validate the results for missing parameters and responses. This can be as simple as raising an exception when it has completed if something is missing, and everything should show up correctly in the gui.

oyvindeide commented 4 months ago

We can do something similar to here: https://github.com/equinor/ert/blob/7fc50c7d1ff401e684a04bb8b8b6cd1f62d08a60/src/ert/analysis/_es_update.py#L153, i.e. load all the observations and check if any are missing, similar to what we do in the update report: https://github.com/equinor/ert/blob/7fc50c7d1ff401e684a04bb8b8b6cd1f62d08a60/src/ert/analysis/snapshots.py#L37

We already validate that the forward model is running as expected, with this we also validate all the observations.

xjules commented 1 week ago

We should get some feedback on forward_model_ok state when hovering over the realization. Let's see if this issue is still valid.

Nevertheless this does not check whether it contains the correct data. Next step: contact @oyvindeide for more info.