search

operator-framework / operator-lifecycle-manager

A management framework for extending Kubernetes with Operators
https://olm.operatorframework.io
Apache License 2.0
1.7k stars 542 forks source link

flakey test causes me to delete fork #2770

Open georgettica opened 2 years ago

georgettica commented 2 years ago

Bug Report

https://github.com/operator-framework/operator-lifecycle-manager/blob/master/.github/workflows/flaky-e2e.yml is super flakey (fails most of the time) and because of that I am deleting my fork

keeping my fork up in case this issue will require you to look at my fork

What did you do? clone the repo

What did you expect to see? that the flaky or any recurring GHA will not break every time it runs.

What did you see instead? Under which circumstances? I saw daily an error and it wasn't clear enough I could PR it and solve.

Environment

N/A

N/A

N/A

Possible Solution N/A

Additional context I rebased the repo multiple times but the same problem persisted

perdasilva commented 2 years ago

Hi @georgettica thank you for the bug report. This is a long and recurring issue and we are trying to address it little-by-little. To address this we've been trying to factor out the flaky tests to a separate (non-required) job - to unblock people and identify the culprits and try to fix them in turn. Over the last few months we've already made several changes to the e2e suite and improved CI a fair bit, though the road to complete stability is long - and we, of course, welcome any help with this effort (see the flaky test issues).

Your frustration is not misplaced and is shared by the other developers of the project. We're doing what we can and appreciate your feedback, patience and, if inclined, help in this matter ^^

georgettica commented 2 years ago

if that is ok, could changes that help be linked to this issue? so if any are frustrated aswell they can vent here and maybe shell out different cards to solve this?

perdasilva commented 2 years ago

@georgettica I'll add here the list of flakes

That would be a great start! If I find other things I'll reference this ticket.

Thanks again for your understanding and patience ^^

georgettica commented 2 years ago

Software can be hard, I feel your pain and hopefully at some point the flakiness will clear up