Tempus_*_MPI_1 failing in Trilinos-atdm-ats1-knl_intel-19.0.4_mpich-7.7.15_openmp_static_opt build starting 2021-10-15

[ZUUL42]

CC: @trilinos/tempus, @ccober6 (Trilinos Nonlinear Solvers / Discretizations Triage/ATDM Contact)

## Next Action Status ## Description As shown in [this query](https://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&begin=2021-10-14&end=2022-1-1&filtercount=6&showfilters=1&filtercombine=and&field1=groupname&compare1=62&value1=Experimental&field2=buildname&compare2=65&value2=Trilinos-atdm-ats1-knl_intel-19.0.4_mpich-7.7.15_openmp_static_opt&field3=testname&compare3=65&value3=Tempus_&field4=testname&compare4=66&value4=_MPI_1&field5=status&compare5=62&value5=passed&field6=testoutput&compare6=95&value6=Floating%20point%20exception) (click "Shown Matching Output" in upper right) the tests: * `Tempus_BDF2_MPI_1` * `Tempus_BackwardEuler_MPI_1` * `Tempus_DIRK_MPI_1` * `Tempus_ExplicitRK_MPI_1` * `Tempus_UnitTest_ERK_MPI_1` in the builds: * `Trilinos-atdm-ats1-knl_intel-19.0.4_mpich-7.7.15_openmp_static_opt` started failing on testing day 2021-10-15. Error: `srun: error: nid00195: task 0: Floating point exception (core dumped)` ## Current Status on CDash Run the [above query](https://testing.sandia.gov/cdash/queryTests.php?project=Trilinos&begin=2021-10-14&end=2022-1-1&filtercount=6&showfilters=1&filtercombine=and&field1=groupname&compare1=62&value1=Experimental&field2=buildname&compare2=65&value2=Trilinos-atdm-ats1-knl_intel-19.0.4_mpich-7.7.15_openmp_static_opt&field3=testname&compare3=65&value3=Tempus_&field4=testname&compare4=66&value4=_MPI_1&field5=status&compare5=62&value5=passed&field6=testoutput&compare6=95&value6=Floating%20point%20exception) adjusting the "Begin" and "End" dates to match today any other date range or just click "CURRENT" in the top bar to see results for the current testing day. ## Steps to Reproduce One should be able to reproduce this failure as described in: * https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md and the system-specific instructions at: * https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md#specific-instructions-for-each-system Just log into any of the associated machines and copy and paste the full CDash build name `` listed above and run commands like: ``` $ cd / $ source $TRILINOS_DIR/cmake/std/atdm/load-env.sh $ cmake \ -GNinja \ -DTrilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/atdm/ATDMDevEnv.cmake \ -DTrilinos_ENABLE_TESTS=ON -DTrilinos_ENABLE_=ON \ $TRILINOS_DIR $ make NP=16 $ ctest -j4 ``` where `` is any package that you want to enable to reproduce build and/or test results. Again, for exact system-specific details on what commands to run to build and run tests, see: * https://github.com/trilinos/Trilinos/blob/develop/cmake/std/atdm/README.md#specific-instructions-for-each-system If you can't figure out what commands to run to reproduce the problem given this documentation, then please post a comment here and we will give you the exact minimal commands.

[ccober6]

@ZUUL42 is this still a problem. I think the above merge should have fixed it.

[ZUUL42]

@ccober6, the CDash query seems to indicate this failure hasn't presented itself since 11/24. So, ya, I think we can call this one closed.