i4Ds / Karabo-Pipeline

The Karabo Pipeline can be used as Digital Twin for SKA
https://i4ds.github.io/Karabo-Pipeline/
MIT License
11 stars 4 forks source link

Fix Flaky Tests #524

Closed Lukas113 closed 8 months ago

Lukas113 commented 10 months ago

Since two months, our scheduled tests don't succeed anymore on a regular basis. However, to me it seems like the tests since the last three weeks work or fail, depending on how the stars are aligned. I'll just list the errors which are flaky and seem to not be fixed yet (since three weeks):

These bugs (especially the first one) needs to be fixed ASAP. They also affect active PR (their tests fail).

Lukas113 commented 10 months ago

I guessed it might have something to do with different installed versions of packages in the environment. However, looking at the diffs of the according runs I don't think so anymore. I haven't found any common denominator.

image image image

Lukas113 commented 8 months ago

The tests don't fail anymore since 1 1/2 months (see CI-Test 880 on 01.12.2023). According to the error message (see image below), I'm pretty sure that the work on the Karabo-Feedstock at the 1. Dec fixed this issue, because since then, the boost-versions of pybdsf are fixed (as it should be according to the conda-doc) and we have pretty much a pybdsf build for each boost-version available.

image