[ci] Intermittent issue when building istio-gateway

DnPlas commented 4 months ago

Bug Description

In a number of CI runs, we have observed an intermittent issue when building istio-gateway.

It is not clear what is causing the issue because the logs in the CI are obscuring out the actual output of charmcraft pack, but we can investigate a bit further. So far I have only been able to see this issue in a PR, it's not happening in my local dev environment.

To Reproduce

Trigger a CI run, either from a PR or from an already merged commit.

Environment

On push/pull CI.

Relevant Log Output

# The first error message we get

 INFO     pytest_operator.plugin:plugin.py:982 Charm build for istio-gateway completed with errors (return code=1) in 175.42s

# Scrolling a bit more into the logs we find this

   File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls/lib/python3.8/site-packages/pytest_operator/plugin.py", line 1004, in build_charm
    raise RuntimeError(
RuntimeError: Failed to build charm charms/istio-gateway:

# Because the logs are mixed with istio-pilot, we see a mixture of both build processes, so for now we cannot pin point exactly where the issue is happening.

Additional Context

We have ruled out that the issue was caused by a change in the code because the PR that triggered this CI run did not change any istio-gateway code or anything that could've affected its build.

syncronize-issues-to-jira[bot] commented 4 months ago

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-5404.

This message was autogenerated

ca-scribner commented 4 months ago

https://github.com/canonical/charmcraft/issues/1456 feels similar, but I dont know if it is the same. I tried debugging for a while and had a hard time with this being intermittent

canonical / istio-operators