Open chapulina opened 3 years ago
I entered the test regression in buildfarmer database and found out that this issue happened less than 25 times. It has happened in the following jobs:
Job | Last time seen | First time seen (Since 2021) |
---|---|---|
ign_transport-ign-8-win | 2023-01-24 | 2021-05-01 |
ign_transport-gz-12-win | 2022-12-01 | 2022-08-31 |
ign_transport-ign-11-win | 2022-09-14 | 2022-05-25 |
ign_transport-ci-win | 2022-08-02 | 2022-08-02 |
This means that this issue is happening on Citadel, Fortress and Garden
This regression is still happening flaky on transport-11 and transport-13
Flaky ratio in the last 30 days:
job_name | last_fail | first_fail | build_count | failure_count | failure_percentage |
---|---|---|---|---|---|
ign_transport-ign-11-win | 2023-10-20 | 2023-10-20 | 7 | 1 | 14.29 |
ign_transport-gz-13-win | 2023-10-17 | 2023-10-08 | 10 | 2 | 20.0 |
FYI @azeey
Flaky ration in the last 30 days:
job_name | last_fail | first_fail | build_count | failure_count | failure_percentage |
---|---|---|---|---|---|
gz_transport-ign-transport11-win | 2024-01-07 | 2023-12-18 | 7 | 2 | 28.57 |
gz_transport-ign-transport8-win | 2024-01-03 | 2024-01-03 | 7 | 1 | 14.29 |
gz_transport-12-win | 2023-12-30 | 2023-12-17 | 7 | 3 | 42.86 |
gz_transport-13-win | 2023-12-26 | 2023-12-12 | 8 | 2 | 25.0 |
This issue seems to be happening more often this week.
Flaky ratio in the last 15 days:
job_name | last_fail | first_fail | build_count | failure_count | failure_percentage |
---|---|---|---|---|---|
gz_transport-main-win | 2024-08-03 | 2024-07-30 | 4 | 2 | 50.0 |
gz_transport-ign-transport8-win | 2024-08-03 | 2024-08-03 | 4 | 1 | 25.0 |
gz_transport-12-win | 2024-07-25 | 2024-07-25 | 4 | 1 | 25.0 |
Log output is almost the same as before:
83: [ RUN ] playback.ReplayStep
83: Stepping playback...
83: C:\J\workspace\gz_transport-main-win\ws\gz-transport\log\test\integration\playback.cc(652): error: Value of: MessagesAreEqual(firstMessageData, secondMessageData)
83: Actual: true
83: Expected: false
83: Waiting to for playback to finish...
83: Done waiting...
83: Playback finished!
83: [ FAILED ] playback.ReplayStep (11265 ms)
This test used to pass consistently, but started failing since 2020-10-23 for
ign-transport8
:https://build.osrfoundation.org/job/ignition_transport-ci-ign-transport8-windows7-amd64/28/testReport/(root)/playback/ReplayStep/history/
It passes on
ign-transport9
though:https://build.osrfoundation.org/job/ignition_transport-ci-ign-transport9-windows7-amd64/1/testReport/(root)/playback/ReplayStep/history/
Failure log: