Closed komish closed 3 months ago
Note that we already have the retry
decorator around the get_run_id
function.
Indeed there's a 30s delay before the test fails, see for example:
2024-05-07T04:08:46.5422908Z Then the user sees the pull request is merged # tests/functional/behave_features/steps/implementation.py:281
2024-05-07T04:09:17.3555539Z Assertion Failed: Workflow for the submitted PR (#None) did not run.
Actually the pr_number
is provided to this function, not populated within this function. So retrying won't solve the issue. The race condition occurs somewhere else.
pr_number
can be None
, this is valid. But in that case we should get secrects.pr_number
instead, as it's done at many other places in the codebaseDebugging:
Starting at tests/functional/behave_features/steps/implementation.py:281
:
@then("the user sees the pull request is merged")
def pull_request_is_merged(context):
context.workflow_test.check_workflow_conclusion(expect_result="success") # This calls chart_certification.py:903
context.workflow_test.check_pull_request_result(expect_merged=True)
context.workflow_test.check_pull_request_labels()
def check_workflow_conclusion(self, expect_result: str):
# Check workflow conclusion
super().check_workflow_conclusion(None, expect_result) # Calls chart_certification.py:333
def check_workflow_conclusion(
self, pr_number, expect_result: str, failure_type="error"
):
# None, "success", "error"
try:
# Check workflow conclusion
run_id = get_run_id(self.secrets, WORKFLOW_CERTIFICATION_CI, pr_number)
[...]
So get_run_id is called with pr_number = None
.
get_run_id
then calls get_pr
, which falls back to using secrets.pr_number
if pr_number is None
(see https://github.com/openshift-helm-charts/development/blob/main/tests/functional/behave_features/common/utils/github.py#L79):
pr_number = secrets.pr_number if pr_number is None else pr_number
As pr_number
is still None
after that, that means that the fallback secrets.pr_number
is None
.
I noticed in a recent smoke test run that the new metadata we're adding to log lines seems to indicate we had an issue finding the expected pull request number for a given workflow run. Log is here.
The full snippet:
I didn't dig into this, but it would seem we have some kind of race condition here, or maybe we're trying to find the pull request for the workflow and not finding it, but also not doing proper error checking.
Creating this issue to make sure it doesn't get lost.