Add a new configuration keyword for memory pr realization

equinor / ert

ERT - Ensemble based Reservoir Tool - is designed for running ensembles of dynamical models such as reservoir models, in order to do sensitivity analysis and data assimilation. ERT supports data assimilation using the Ensemble Smoother (ES), Ensemble Smoother with Multiple Data Assimilation (ES-MDA) and Iterative Ensemble Smoother (IES).

https://ert.readthedocs.io/en/latest/

GNU General Public License v3.0

98 stars 104 forks source link

Add a new configuration keyword for memory pr realization #8215

Open berland opened 1 week ago

berland commented 1 week ago

Issue Resolves #7798

Approach Adding a new keyword that is propagated to via QueueConfig to driver.submit(). Overlaps with Torque queue option MEMORY_PER_JOB which should be deprecated if this PR goes through.

[x] PR title captures the intent of the changes, and is fitting for release notes.
[ ] Added appropriate release note label
[ ] Commit history is consistent and clean, in line with the contribution guidelines.
[ ] Make sure tests pass locally (after every commit!)

When applicable

[ ] When there are user facing changes: Updated documentation
[ ] New behavior or changes to existing untested code: Ensured that unit tests are added (See Ground Rules).
[ ] Large PR: Prepare changes in small commits for more convenient review
[ ] Bug fix: Add regression test for the bug
[ ] Bug fix: Create Backport PR to latest release

codecov-commenter commented 1 week ago

Codecov Report

Attention: Patch coverage is 97.14286% with 1 line in your changes missing coverage. Please review.

Project coverage is 87.03%. Comparing base (a9db741) to head (5ad7951). Report is 7 commits behind head on main.

Files	Patch %	Lines
src/ert/scheduler/lsf_driver.py	88.88%	1 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #8215 +/- ## ======================================= Coverage 87.02% 87.03% ======================================= Files 378 378 Lines 23955 23986 +31 Branches 618 634 +16 ======================================= + Hits 20848 20876 +28 - Misses 3026 3037 +11 + Partials 81 73 -8 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

berland commented 1 week ago

todo:

catch overspecification in config system
solve conflict with the auto-inserted memory usage from ert-configurations

berland commented 1 week ago

catch overspecification in config system

Fixed

solve conflict with the auto-inserted memory usage from ert-configurations

Fixed by doing silent replacement for the LSF driver when mem= is already present in the resource requirement string.