In the previous PR updating the reference output of the FEniCS-FEniCS test: https://github.com/precice/systemtests/pull/266 the update was done by running the following configuration on a local machine:
It was expected that once the locally updated results are updated as reference output, the system test will pass. That however did not happen. Further investigation reveals that the VTK output generated from the CI pipeline varies slightly with the VTK output obtained locally. The variation is close to machine precision (order of E-12). This variation can be a result of using different versions of dependencies in the CI pipeline. The variation is small enough to be ignored.
The following strategy is applied to update the VTK output of the FEniCS-FEniCS test:
The results of the test are pushed to precice_st_output using the push.py script. By adding the -o or --output flag to the execution of this push.py script, the VTK output can be dumped in the precice_st_output repository. (@Eder-K thanks for this feature!)
The VTK output dumped in precice_st_output is then copied as the reference output for the system test. This ensures that the output exactly matches.
In the previous PR updating the reference output of the FEniCS-FEniCS test: https://github.com/precice/systemtests/pull/266 the update was done by running the following configuration on a local machine:
It was expected that once the locally updated results are updated as reference output, the system test will pass. That however did not happen. Further investigation reveals that the VTK output generated from the CI pipeline varies slightly with the VTK output obtained locally. The variation is close to machine precision (order of E-12). This variation can be a result of using different versions of dependencies in the CI pipeline. The variation is small enough to be ignored.
The following strategy is applied to update the VTK output of the FEniCS-FEniCS test:
-oor--outputflag to the execution of thispush.pyscript, the VTK output can be dumped in theprecice_st_outputrepository. (@Eder-K thanks for this feature!)precice_st_outputis then copied as the reference output for the system test. This ensures that the output exactly matches.