Hard to reproduce test failures

[susilehtola]

In addition to #64, I am also seeing other types of test failures. The key-MS test is a case which often fails with the following pattern

 79/114 Test  #83: key-MS ...........................***Failed    0.19 sec
          MOPAC Job: "MS.mop" ended normally on Jun 21, 2022, at 18:40.
WARNING: output file size mismatch, 1032 vs. 1024
Traceback (most recent call last):
  File "/builddir/build/BUILD/mopac-22.0.2/tests/keywords/../run_test.py", line 32, in <module>
    compare_mopac_out_file(out_line, out_list, ref_line, ref_list)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/mopac-22.0.2/tests/compare_output.py", line 225, in compare_mopac_out_file
    assert ref == out, f'ERROR: string mismatch between {ref} on reference line {ref_line0} and {out} on output line {out_line0}'
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: ERROR: string mismatch between GRADIENT on reference line 388 and NO. on output line 388

Unfortunately, I have not been able to access the log files for the failing runs.

[godotalgorithm]

When I was first putting together the CTest/Python-based testing framework, I made some changes to MOPAC itself to suppress some "erratic" outputs that depended on small timing differences between machines (i.e. outputs that only appear if a calculation was performed sufficiently slowly). I also have regex expressions to filter out machine-dependent outputs like timing information. There may still be such things remaining in the code, and I should either fix them or adjust the regex expressions in the testing process to ignore them. This may have to occur on a case-by-case basis, so keep me informed of test failures that you see as I fix the known failures and known common causes.

[godotalgorithm]

While I've closed this Issue for now, feel free to reopen it if my PR doesn't actually fix the problem, which I am unable to directly reproduce.

[susilehtola]

Test still fails after #70 and #71

          MOPAC Job: "MS.mop" ended normally on Jun 24, 2022, at 17:14.
WARNING: output file size mismatch, 1032 vs. 1024
Traceback (most recent call last):
  File "/builddir/build/BUILD/mopac-22.0.2/tests/run_test.py", line 32, in <module>
    compare_mopac_out_file(out_line, out_list, ref_line, ref_list)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/mopac-22.0.2/tests/compare_output.py", line 225, in compare_mopac_out_file
    assert ref == out, f'ERROR: string mismatch between {ref} on reference line {ref_line0} and {out} on output line {out_line0}'
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: ERROR: string mismatch between GRADIENT on reference line 388 and NO. on output line 388

[godotalgorithm]

I'll add "GRADIENT NORM" to the regex filter in the testing script, and it will just be ignored for now.