search

cadet / CADET-Core

A modular, fast, and accurate simulation framework for (bio)chemical processes
Other
65 stars 26 forks source link

Non-reproducable CI results #273

Closed jbreue16 closed 1 week ago

jbreue16 commented 2 months ago

We suspect that our CI sometimes produces different results that are not always reproducable. We should keep track of that and can link CI runs here in this issue to discuss potential causes.

Update: This is also an issue on windows, see the full CI run compared to running the specific test that failed locally running the CI tests on windows now also fails for one test

jbreue16 commented 2 months ago

I found evidence that the CI is non-reproducable: Exactly one of the reference tests did not run here: https://github.com/cadet/CADET-Core/actions/runs/10421401942/job/28863536531 Disabling that test, another test all of a sudden stopped working, even though it worked in the previous run above: https://github.com/cadet/CADET-Core/actions/runs/10407694093/job/28847668469

On top of that, the test which failed in https://github.com/cadet/CADET-Core/actions/runs/10421401942/job/28863536531 works just fine when run as the only CI test: https://github.com/cadet/CADET-Core/actions/runs/10422346048

ronald-jaepel commented 1 month ago

Here are my current results:

From 4 weeks ago:

I don't remember which branch these are from. I think fix/tests

new data from 20.8.:

branch fit/tests

All work

branch master:

Running testRunner [CI] completes without errors.