Intermittent fortran errors

modflowpy / flopy

A Python package to create, run, and post-process MODFLOW-based models.

https://flopy.readthedocs.io

Other

517 stars 313 forks source link

Intermittent fortran errors #1490

Closed wpbonelli closed 2 months ago

wpbonelli commented 2 years ago

I occasionally see fortran errors when running the test suite. These have occurred locally and in CI. ~~I am pretty sure Windows-only.~~ Update: seen on both Windows and Mac.

Will continue to add examples from CI runs as they occur.

Affected tests include:

test_lgr.py::test_simple_lgrmodel_from_scratch: forrtl: error (65): floating invalid
- https://github.com/w-bonelli/flopy/runs/7744805897?check_suite_focus=true#step:8:1832
test_usg.py::test_freyberg_usg: forrtl: error (72): floating overflow
test_mt3d.py::test_mfnwt_CrnkNic: forrtl: error (72): floating overflow, forrtl: error (74): floating underflow
- https://github.com/modflowpy/flopy/runs/7808967872?check_suite_focus=true#step:7:3464
- https://github.com/modflowpy/flopy/runs/7848124449?check_suite_focus=true#step:9:3646

Currently using the flaky pytest plugin to retry on failure.

mwtoews commented 2 years ago

These are tricky errors. A similar bug https://github.com/MODFLOW-USGS/mt3d-usgs/pull/46 was fixed with a debug build and inspecting the routines and variables near the crash (I probably used gdb for that one). The bug probably only shows on Windows because how the compiler implicitly allocates memory, but is still a bug.

langevin-usgs commented 2 years ago

We love to get to the bottom of these little bugs, but they are sometimes hard to reproduce. There are a couple of intermittent Fortran bugs that show up on CI from time to time. If they are in older legacy versions of MODFLOW, then they are lower priority, but it's good to call attention to them.

wpbonelli commented 2 years ago

Would it be best to close this and keep track of these in another location? I guess it sounds like they are not really flopy issues. I wondered if I might have introduced some of them with the recent pytest changes. Apologies, should have asked before filing an issue.

langevin-usgs commented 2 years ago

It's good that you raised this here as they are affecting our CI. It would probably be good to keep track of offending tests like this that fail intermittently and also those that take a long time to run. We can then assign people to look into them and see if we can get some resolution.

wpbonelli commented 2 months ago

I opened corresponding issues on the usg and mt3d repos so we don't lose track of these, I'll close here as these aren't flopy problems.