Attempt to automatically check for result modification between Mmg runs

[Algiane]

Attempt to reduce the time passed in the post-treatment of the continuous integration.

In its current state, the continuous integration of Mmg checks only that Mmg runs smoothly but it doesn't check if remeshing results have been modified and in this case if they have been improved or degraded.

Idea

This branch addresses the issue of the check of unexpected results modifications and let the judgement of the improvement/degradation to the developper:

• if a commit message contains the [skip output comparison] pattern, then the continuous integration runs as previously and checks the exit value of the execution of each test

• otherwise, the display of timers and mmg version are disabled (precompilation flag MMG_COMPARABLE_OUTPUT) and the continous integration fails if the result of a given test case is different from a reference result.

  Implementation

  ctest allows to perform only one command per test and we want to: (1) run the Mmg test, (2) redirect the output inside a file, (3) compare this output file to reference output. Thus:

  1. CMake generates automatically a first script named <testname>.cmake in the <build_dir>/cmake_scripts directory:

     • such a scripts wraps in an EXECUTE_PROCESS CMake command (to pipe multiple terminal commands):

       • the test case command line
       • the copy of the terminal output in a specific output file

     • the EXECUTE_PROCESS command fails if any of previous commands are failing.

  2. a new test that calls the previous script is added (so it fails if the test case runtime fails)

3. CMake generates automatically a second script named <testname>_compare_to_ref.cmake in the <build_dir>/cmake_scripts directory:

   • this scripts wraps in an EXECUTE_PROCESS CMake command (in order to choose the failure criterion):

     • the comparison of the previous terminal output to a reference output (available for example in the test case input folder)
     • the redirection of the terminal output in a specific file (optional)

   • the EXECUTE_PROCESS command fails:

     • for a non empty diff if results have to be compared
     • if output redirection fails in case where we want to skip the output comparison (in this case the execute_process should not fail in fact).

4. A second test is added to submit this second script.

5. Previous steps are implemented in the CMake macro MMG_ADD_TEST called in mmg<X>_tests.cmake (instead of the ADD_TEST command)

   Limitations

   This dev was abandoned due to :

   • the handling of the different results with and without scotch renumbering (mainly)
   • the time to update reference results in the test case server in case of algo modifications (to a lesser extent)

Remark

Useful commits of this branch have been cherry-picked in the feature/improve-ci-tokeep branch and integrated to Mmg.

[Algiane]

This PR has been done to keep track of the work of this branch but cannot be integrated as is and there is no plane to improve it for now.