Flake: waiting for catalog pod to be available (for sync) - IDLE

[timflannagan]

This is flake is occasionally consistent throughout workflow runs, but difficult to reproduce locally. Link to a GHA workflow run that contains this: https://github.com/operator-framework/operator-lifecycle-manager/pull/2367/checks?check_run_id=3760863789.

There are a couple of test instances where this issue has propagated:

• User defined service account/with permission
• User defined service account/with no permission
• Operator Group/ static provider
• Install Plan/with CRD schema change/existing version is present in new CRD (deprecated field)

Creating catalog source scoped-catsrc-785ll in namespace scoped-ns-xrfbn...
Catalog source scoped-catsrc-785ll created
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
...
• Failure [300.404 seconds]
User defined service account
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/user_defined_sa_test.go:25
  with permission [It]
  /home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/user_defined_sa_test.go:82

    Error Trace:    user_defined_sa_test.go:110
                                runner.go:113
                                runner.go:64
                                it_node.go:26
                                spec.go:215
                                spec.go:138
                                spec_runner.go:200
                                spec_runner.go:170
                                spec_runner.go:66
                                suite.go:79
                                ginkgo_dsl.go:238
                                ginkgo_dsl.go:221
                                e2e_test.go:57
    Error:          Received unexpected error:
                    timed out waiting for the condition
    Test:           User defined service account with permission

[timflannagan]

cc @dinhxuanvu