search

canonical / bundle-kubeflow

Charmed Kubeflow
Apache License 2.0
104 stars 50 forks source link

ci(main): bundle tests fail to build charms with zipfile EOFError #1005

Closed orfeas-k closed 2 months ago

orfeas-k commented 6 months ago

Bug Description

As of yesterday, bundle tests v2 fail to build the charms with the error shown in logs below. This has been observed in multiple runs (example) with the same exact logs. Note that:

To Reproduce

Run the CI from main.

Environment

Juju 3.5 Microk8s v1.25.16 charmcraft 2.6.0

Relevant Log Output

logs from running August 2024

::    ::   Successfully built setuptools wheel
::    ::   Installing collected packages: setuptools, wheel
::    ::   ERROR: Exception:
::    ::   Traceback (most recent call last):
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
::    ::       status = self.run(options, args)
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 421, in run
::    ::       installed = install_given_reqs(
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/req/__init__.py", line 67, in install_given_reqs
::    ::       requirement.install(
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 809, in install
::    ::       install_wheel(
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/operations/install/wheel.py", line 606, in install_wheel
::    ::       unpack_file(wheel_path, unpacked_dir.path)
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/utils/unpacking.py", line 249, in unpack_file
::    ::       unzip_file(
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/utils/unpacking.py", line 139, in unzip_file
::    ::       shutil.copyfileobj(fp, destfp)
::    ::     File "/usr/lib/python3.8/shutil.py", line 205, in copyfileobj
::    ::       buf = fsrc_read(length)
::    ::     File "/usr/lib/python3.8/zipfile.py", line 940, in read
::    ::       data = self._read1(n)
::    ::     File "/usr/lib/python3.8/zipfile.py", line 1008, in _read1
::    ::       data += self._read2(n - len(data))
::    ::     File "/usr/lib/python3.8/zipfile.py", line 1043, in _read2
::    ::       raise EOFError
::    ::   EOFError
::    ::   ----------------------------------------
::    :: ERROR: Command errored out with exit status 2: /root/parts/charm/build/staging-venv/bin/python3 /root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-_x_j7juu/overlay --no-warn-script-location --no-binary anyio,attrs,certifi,charmed-kubeflow-chisme,charset-normalizer,deepdiff,exceptiongroup,h11,httpcore,httpx,idna,importlib-resources,jsonschema,lightkube,lightkube-models,markupsafe,ops,ordered-set,pip,pkgutil-resolve-name,pyrsistent,pyyaml,requests,ruamel-yaml,ruamel-yaml-clib,serialized-data-interface,setuptools,sniffio,tenacity,urllib3,websocket-client,wheel,zipp --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.8.0' wheel Check the logs for full command output.
:: WARNING: Initial package installation failed. Falling back to older method, which may leave your charm in an un-runnable state.
:: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--upgrade', 'jinja2']
::    :: Collecting jinja2
::    ::   Downloading jinja2-3.1.4-py3-none-any.whl (133 kB)
::    :: Collecting MarkupSafe>=2.0
::    ::   Downloading MarkupSafe-2.1.5-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26 kB)
::    :: Installing collected packages: MarkupSafe, jinja2
::    :: Successfully installed MarkupSafe-2.1.5 jinja2-3.1.4
:: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--upgrade', '--no-binary', ':all:', 'wheel', 'setuptools', 'pip']
::    :: Collecting wheel
::    ::   Using cached wheel-0.43.0.tar.gz (99 kB)
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::     Preparing wheel metadata: started
::    ::     Preparing wheel metadata: finished with status 'done'
::    :: Collecting setuptools
::    ::   Using cached setuptools-69.5.1.tar.gz (2.3 MB)
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::   Installing backend dependencies: started
::    ::   Installing backend dependencies: finished with status 'done'
::    ::     Preparing wheel metadata: started
::    ::     Preparing wheel metadata: finished with status 'done'
::    :: Collecting pip
::    ::   Downloading pip-24.0.tar.gz (2.1 MB)
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::     Preparing wheel metadata: started
::    ::     Preparing wheel metadata: finished with status 'done'
::    :: Building wheels for collected packages: wheel, setuptools, pip
::    ::   Building wheel for wheel (PEP 517): started
::    ::   Building wheel for wheel (PEP 517): finished with status 'done'
::    ::   Created wheel for wheel: filename=wheel-0.43.0-py3-none-any.whl size=65775 sha256=55c570405f142630c6b9f72fe09d9b67cf1477fcf543ae5b8dcb1f5b7377da81
::    ::   Stored in directory: /root/.cache/pip/wheels/d4/a3/b2/6faade3bb04cd720bb432a6c21ec7baa2542801334a4d47a66
::    ::   Building wheel for setuptools (PEP 517): started
::    ::   Building wheel for setuptools (PEP 517): finished with status 'done'
::    ::   Created wheel for setuptools: filename=setuptools-69.5.1-py3-none-any.whl size=894566 sha256=075e940cfa3659352ca81f87fabf8b4b75ccdbb5f99f2a71a4f77df1b6707fbc
::    ::   Stored in directory: /root/.cache/pip/wheels/57/b6/41/311c955c1ca3eed447666adf5ca9680ddcdd11dc785a746bf6
::    ::   Building wheel for pip (PEP 517): started
::    ::   Building wheel for pip (PEP 517): finished with status 'done'
::    ::   Created wheel for pip: filename=pip-24.0-py3-none-any.whl size=2110223 sha256=7176c5cddbc465dc78dddac6545a4635901a7c9a08a959f4f3bbb3ca0bee944a
::    ::   Stored in directory: /root/.cache/pip/wheels/c1/d6/d4/2f7765d82b17d8c25425875b2dce5af9f22746668c3add50f6
::    :: Successfully built wheel setuptools pip
::    :: Installing collected packages: wheel, setuptools, pip
::    ::   Attempting uninstall: setuptools
::    ::     Found existing installation: setuptools 44.0.0
::    ::     Uninstalling setuptools-44.0.0:
::    ::       Successfully uninstalled setuptools-44.0.0
::    ::   Attempting uninstall: pip
::    ::     Found existing installation: pip 20.0.2
::    ::     Uninstalling pip-20.0.2:
::    ::       Successfully uninstalled pip-20.0.2
::    :: Successfully installed pip-24.0 setuptools-69.5.1 wheel-0.43.0
:: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--upgrade', '--no-binary', ':all:', '--requirement=requirements.txt']
::    :: Collecting anyio==3.7.1 (from -r requirements.txt (line 7))
::    ::   Downloading anyio-3.7.1.tar.gz (142 kB)
::    ::      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 142.9/142.9 kB 3.9 MB/s eta 0:00:00
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::   Installing backend dependencies: started
::    ::   Installing backend dependencies: finished with status 'done'
::    ::   Preparing metadata (pyproject.toml): started
::    ::   Preparing metadata (pyproject.toml): finished with status 'done'
::    :: Collecting attrs==23.1.0 (from -r requirements.txt (line 9))
::    ::   Downloading attrs-23.1.0.tar.gz (212 kB)
::    ::      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 212.9/212.9 kB 29.5 MB/s eta 0:00:00
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'error'
::    ::   error: subprocess-exited-with-error
::    ::
::    ::   × pip subprocess to install build dependencies did not run successfully.
::    ::   │ exit code: 1
::    ::   ╰─> [103 lines of output]
::    ::       Collecting hatchling
::    ::         Downloading hatchling-1.24.2.tar.gz (63 kB)
::    ::            ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.0/64.0 kB 2.0 MB/s eta 0:00:00
::    ::         Getting requirements to build wheel: started
::    ::         Getting requirements to build wheel: finished with status 'done'
::    ::         Installing backend dependencies: started
::    ::         Installing backend dependencies: finished with status 'error'
::    ::         error: subprocess-exited-with-error
::    ::
::    ::         × pip subprocess to install backend dependencies did not run successfully.
::    ::         │ exit code: 1
::    ::         ╰─> [81 lines of output]
::    ::             Collecting packaging>=23.2
::    ::               Using cached packaging-24.0-py3-none-any.whl
::    ::             Collecting pathspec>=0.10.1
::    ::               Downloading pathspec-0.12.1.tar.gz (51 kB)
::    ::                  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.0/51.0 kB 895.3 kB/s eta 0:00:00
::    ::               Installing build dependencies: started
::    ::               Installing build dependencies: finished with status 'done'
::    ::               Getting requirements to build wheel: started
::    ::               Getting requirements to build wheel: finished with status 'done'
::    ::               Preparing metadata (pyproject.toml): started
::    ::               Preparing metadata (pyproject.toml): finished with status 'done'
::    ::             Collecting trove-classifiers
::    ::               Downloading trove-classifiers-2024.4.10.tar.gz (16 kB)
::    ::               Installing build dependencies: started
::    ::               Installing build dependencies: finished with status 'done'
::    ::               Getting requirements to build wheel: started
::    ::               Getting requirements to build wheel: finished with status 'done'
::    ::               Installing backend dependencies: started
::    ::               Installing backend dependencies: finished with status 'done'
::    ::               Preparing metadata (pyproject.toml): started
::    ::               Preparing metadata (pyproject.toml): finished with status 'done'
::    ::             Collecting tomli>=1.2.2
::    ::               Using cached tomli-2.0.1-py3-none-any.whl
::    ::             Collecting pluggy>=1.0.0
::    ::               Downloading pluggy-1.5.0.tar.gz (67 kB)
::    ::                  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 68.0/68.0 kB 3.5 MB/s eta 0:00:00
::    ::               Installing build dependencies: started
::    ::               Installing build dependencies: finished with status 'error'
::    ::               error: subprocess-exited-with-error
::    ::
::    ::               × pip subprocess to install build dependencies did not run successfully.
::    ::               │ exit code: 2
::    ::               ╰─> [38 lines of output]
::    ::                   Collecting setuptools>=45.0
::    ::                     Using cached setuptools-69.5.1-py3-none-any.whl
::    ::                   Collecting setuptools-scm>=6.2.3 (from setuptools-scm[toml]>=6.2.3)
::    ::                     Using cached setuptools_scm-8.1.0-py3-none-any.whl
::    ::                   Collecting packaging>=20 (from setuptools-scm>=6.2.3->setuptools-scm[toml]>=6.2.3)
::    ::                     Using cached packaging-24.0-py3-none-any.whl
::    ::                   Collecting typing-extensions (from setuptools-scm>=6.2.3->setuptools-scm[toml]>=6.2.3)
::    ::                     Using cached typing_extensions-4.11.0-py3-none-any.whl
::    ::                   Collecting tomli>=1 (from setuptools-scm>=6.2.3->setuptools-scm[toml]>=6.2.3)
::    ::                     Using cached tomli-2.0.1-py3-none-any.whl
::    ::                   Installing collected packages: typing-extensions, tomli, setuptools, packaging, setuptools-scm
::    ::                   ERROR: Exception:
::    ::                   Traceback (most recent call last):
::    ::                     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
::    ::                       status = run_func(*args)
::    ::                     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
::    ::                       return func(self, options, args)
::    ::                     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 452, in run
::    ::                       installed = install_given_reqs(
::    ::                     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/req/__init__.py", line 72, in install_given_reqs
::    ::                       requirement.install(
::    ::                     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 856, in install
::    ::                       install_wheel(
::    ::                     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/operations/install/wheel.py", line 725, in install_wheel
::    ::                       _install_wheel(
::    ::                     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/operations/install/wheel.py", line 585, in _install_wheel
::    ::                       file.save()
::    ::                     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/operations/install/wheel.py", line 384, in save
::    ::                       shutil.copyfileobj(f, dest)
::    ::                     File "/usr/lib/python3.8/shutil.py", line 205, in copyfileobj
::    ::                       buf = fsrc_read(length)
::    ::                     File "/usr/lib/python3.8/zipfile.py", line 940, in read
::    ::                       data = self._read1(n)
::    ::                     File "/usr/lib/python3.8/zipfile.py", line 1008, in _read1
::    ::                       data += self._read2(n - len(data))
::    ::                     File "/usr/lib/python3.8/zipfile.py", line 1043, in _read2
::    ::                       raise EOFError
::    ::                   EOFError
::    ::                   [end of output]
::    ::
::    ::               note: This error originates from a subprocess, and is likely not a problem with pip.
::    ::             error: subprocess-exited-with-error
::    ::
::    ::             × pip subprocess to install build dependencies did not run successfully.
::    ::             │ exit code: 2
::    ::             ╰─> See above for output.
::    ::
::    ::             note: This error originates from a subprocess, and is likely not a problem with pip.
::    ::             [end of output]
::    ::
::    ::         note: This error originates from a subprocess, and is likely not a problem with pip.
::    ::       error: subprocess-exited-with-error
::    ::
::    ::       × pip subprocess to install backend dependencies did not run successfully.
::    ::       │ exit code: 1
::    ::       ╰─> See above for output.
::    ::
::    ::       note: This error originates from a subprocess, and is likely not a problem with pip.
::    ::       [end of output]
::    ::
::    ::   note: This error originates from a subprocess, and is likely not a problem with pip.
::    :: error: subprocess-exited-with-error
::    ::
::    :: × pip subprocess to install build dependencies did not run successfully.
::    :: │ exit code: 1
::    :: ╰─> See above for output.
::    ::
::    :: note: This error originates from a subprocess, and is likely not a problem with pip.
:: Traceback (most recent call last):
::   File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 260, in _install_dependencies
::     _process_run(
::   File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 408, in _process_run
::     raise RuntimeError(f"Subprocess command {cmd} execution failed with retcode {retcode}")
:: RuntimeError: Subprocess command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=anyio,attrs,certifi,charmed-kubeflow-chisme,charset-normalizer,deepdiff,exceptiongroup,h11,httpcore,httpx,idna,importlib-resources,jsonschema,lightkube,lightkube-models,markupsafe,ops,ordered-set,pip,pkgutil-resolve-name,pyrsistent,pyyaml,requests,ruamel-yaml,ruamel-yaml-clib,serialized-data-interface,setuptools,sniffio,tenacity,urllib3,websocket-client,wheel,zipp', '--requirement=requirements.txt', 'pip', 'setuptools', 'wheel'] execution failed with retcode 1
::
:: During handling of the above exception, another exception occurred:
::
:: Traceback (most recent call last):
::   File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 477, in <module>
::     main()
::   File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 472, in main
::     builder.build_charm()
::   File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 101, in build_charm
::     self.handle_dependencies()
::   File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 354, in handle_dependencies
::     self._install_dependencies(staging_venv_dir)
::   File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/instrum.py", line 152, in _f
::     return func(*args, **kwargs)
::   File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 289, in _install_dependencies
::     _process_run(cmd)
::   File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 408, in _process_run
::     raise RuntimeError(f"Subprocess command {cmd} execution failed with retcode {retcode}")
:: RuntimeError: Subprocess command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--upgrade', '--no-binary', ':all:', '--requirement=requirements.txt'] execution failed with retcode 1
Parts processing error: Failed to run the build script for part 'charm'.
Failed to build charm for bases index '0'.
Full execution log: '/home/runner/.local/state/charmcraft/log/charmcraft-20240509-152917.976217.log'

Additional Context

No response

syncronize-issues-to-jira[bot] commented 6 months ago

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-5656.

This message was autogenerated

orfeas-k commented 6 months ago

Although this failed for 4 consecutive times, reran this now and it succeeded https://github.com/canonical/kfp-operators/actions/runs/9019490085/job/24814815779?pr=456.

orfeas-k commented 6 months ago

This has also happened in v1 tests actually

orfeas-k commented 5 months ago

We see this in different charms as well e.g. istio-pilot

Full logs ``` Added 'test-istio' model on microk8s/localhost with credential 'microk8s' for user 'admin' integration-tls-provider: install_deps> python -I -m pip install -r requirements-integration.txt integration-tls-provider: freeze> python -m pip freeze --all integration-tls-provider: aiohttp==3.[8](https://github.com/canonical/istio-operators/actions/runs/9384901845/job/25841748232?pr=434#step:4:9).5,aiosignal==1.3.1,anyio==4.0.0,asttokens==2.4.0,async-timeout==4.0.3,asyncio==3.4.3,attrs==23.1.0,backcall==0.2.0,bcrypt==4.0.1,beautifulsoup4==4.12.2,cachetools==5.3.1,certifi==2023.7.22,cffi==1.15.1,charset-normalizer==3.2.0,cryptography==41.0.3,decorator==5.1.1,exceptiongroup==1.1.3,executing==1.2.0,frozenlist==1.4.0,google-auth==2.22.0,h11==0.14.0,httpcore==0.17.3,httpx==0.24.1,hvac==1.2.0,idna==3.4,iniconfig==2.0.0,ipdb==0.13.13,ipython==8.12.2,jedi==0.1[9](https://github.com/canonical/istio-operators/actions/runs/9384901845/job/25841748232?pr=434#step:4:10).0,Jinja2==3.1.2,juju==3.4.0.0,kubernetes==27.2.0,lightkube==0.14.0,lightkube-models==1.28.1.4,macaroonbakery==1.3.1,MarkupSafe==2.1.3,matplotlib-inline==0.1.6,multidict==6.0.4,mypy-extensions==1.0.0,oauthlib==3.2.2,packaging==23.1,paramiko==2.12.0,parso==0.8.3,pexpect==4.8.0,pickleshare==0.7.5,pip==24.0,pluggy==1.3.0,prompt-toolkit==3.0.39,protobuf==3.20.3,ptyprocess==0.7.0,pure-eval==0.2.2,pyasn1==0.5.0,pyasn1-modules==0.3.0,pycparser==2.21,Pygments==2.16.1,pyhcl==0.4.5,pymacaroons==0.13.0,PyNaCl==1.5.0,pyRFC3339==1.1,pytest==7.4.2,pytest-asyncio==0.21.1,pytest-operator==0.29.0,python-dateutil==2.8.2,pytz==2023.3.post1,PyYAML==6.0.1,requests==2.31.0,requests-oauthlib==1.3.1,rsa==4.9,setuptools==69.5.1,six==1.16.0,sniffio==1.3.0,soupsieve==2.5,stack-data==0.6.2,tenacity==8.2.3,tomli==2.0.1,toposort==1.[10](https://github.com/canonical/istio-operators/actions/runs/9384901845/job/25841748232?pr=434#step:4:11),traitlets==5.9.0,typing-inspect==0.9.0,typing_extensions==4.7.1,urllib3==1.26.16,wcwidth==0.2.6,websocket-client==1.6.2,websockets==8.1,wheel==0.43.0,yarl==1.9.2 integration-tls-provider: commands[0]> pytest --show-capture=no --log-cli-level=INFO -vvs --tb=native --model test-istio tests/test_bundle_tls_provider.py ============================= test session starts ============================== platform linux -- Python 3.8.10, pytest-7.4.2, pluggy-1.3.0 -- /home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/bin/python cachedir: .tox/integration-tls-provider/.pytest_cache rootdir: /home/runner/work/istio-operators/istio-operators configfile: pyproject.toml plugins: anyio-4.0.0, asyncio-0.21.1, operator-0.29.0 asyncio: mode=strict collecting ... collected 2 items tests/test_bundle_tls_provider.py::test_build_and_deploy_istio_charms -------------------------------- live log setup -------------------------------- INFO pytest_operator.plugin:plugin.py:675 Connecting to existing model github-pr-8e71e-microk8s:test-istio on unspecified cloud -------------------------------- live log call --------------------------------- INFO pytest_operator.plugin:plugin.py:526 Using tmp_path: /home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/tmp/pytest/test-istio0 INFO pytest_operator.plugin:plugin.py:975 Building charm istio-gateway INFO pytest_operator.plugin:plugin.py:526 Using tmp_path: /home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/tmp/pytest/test-istio0 INFO pytest_operator.plugin:plugin.py:975 Building charm istio-pilot INFO pytest_operator.plugin:plugin.py:982 Charm build for istio-gateway completed with errors (return code=1) in 156.92s FAILED tests/test_bundle_tls_provider.py::test_tls_configuration XFAIL ------------------------------ live log teardown ------------------------------- INFO pytest_operator.plugin:plugin.py:783 Model status: Model Controller Cloud/Region Version SLA Timestamp test-istio github-pr-8e71e-microk8s microk8s/localhost 3.5.1 unsupported 13:28:51Z INFO pytest_operator.plugin:plugin.py:789 Juju error logs: INFO pytest_operator.plugin:plugin.py:855 Forgetting main... =================================== FAILURES =================================== ______________________ test_build_and_deploy_istio_charms ______________________ Traceback (most recent call last): File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/_pytest/runner.py", line 341, in from_call result: Optional[TResult] = func() File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/_pytest/runner.py", line 262, in lambda: ihook(item=item, **kwds), when=when, reraise=reraise File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/pluggy/_hooks.py", line 493, in __call__ return self._hookexec(self.name, self._hookimpls, kwargs, firstresult) File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/pluggy/_manager.py", line [11](https://github.com/canonical/istio-operators/actions/runs/9384901845/job/25841748232?pr=434#step:4:12)5, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/pluggy/_callers.py", line 152, in _multicall return outcome.get_result() File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/pluggy/_result.py", line 114, in get_result raise exc.with_traceback(exc.__traceback__) File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/pluggy/_callers.py", line 77, in _multicall res = hook_impl.function(*args) File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/_pytest/runner.py", line 177, in pytest_runtest_call raise e File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/_pytest/runner.py", line 169, in pytest_runtest_call item.runtest() File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/_pytest/python.py", line 1792, in runtest self.ihook.pytest_pyfunc_call(pyfuncitem=self) File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/pluggy/_hooks.py", line 493, in __call__ return self._hookexec(self.name, self._hookimpls, kwargs, firstresult) File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/pluggy/_manager.py", line 115, in _hookexec return self._inner_hookexec(hook_name, methods, kwargs, firstresult) File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/pluggy/_callers.py", line 152, in _multicall return outcome.get_result() File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/pluggy/_result.py", line 114, in get_result raise exc.with_traceback(exc.__traceback__) File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/pluggy/_callers.py", line 77, in _multicall res = hook_impl.function(*args) File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/_pytest/python.py", line 194, in pytest_pyfunc_call result = testfunction(**testargs) File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/pytest_asyncio/plugin.py", line 532, in inner _loop.run_until_complete(task) File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete return future.result() File "/home/runner/work/istio-operators/istio-operators/tests/test_bundle_tls_provider.py", line 33, in test_build_and_deploy_istio_charms istio_charms = await ops_test.build_charms(f"{charms_path}-gateway", f"{charms_path}-pilot") File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/pytest_operator/plugin.py", line 1021, in build_charms charms = await asyncio.gather( File "/home/runner/work/istio-operators/istio-operators/.tox/integration-tls-provider/lib/python3.8/site-packages/pytest_operator/plugin.py", line 1004, in build_charm raise RuntimeError( RuntimeError: Failed to build charm charms/istio-gateway: Packing the charm. Launching environment to pack for base name='ubuntu' channel='20.04' architectures=['amd64'] (may take a while the first time but it's reusable) Packing the charm Packing the charm. Building charm in '/root' Running step PULL for part 'charm' Execute action Running step BUILD for part 'charm' Execute action :: + env -i LANG=C.UTF-8 LC_ALL=C.UTF-8 PATH=/snap/charmcraft/3454/libexec/charmcraft:/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin SNAP=/snap/charmcraft/3454 SNAP_ARCH=amd64 SNAP_NAME=charmcraft SNAP_VERSION=2.6.0 /snap/charmcraft/3454/bin/python3 -u -I /snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py --builddir /root/parts/charm/build --installdir /root/parts/charm/install --entrypoint /root/parts/charm/build/src/charm.py -p wheel -p setuptools -p pip -r requirements.txt :: Starting charm builder :: Translated .jujuignore 2 '.git' => '.*/\\.git\\Z' :: Translated .jujuignore 3 '.svn' => '.*/\\.svn\\Z' :: Translated .jujuignore 4 '.hg' => '.*/\\.hg\\Z' :: Translated .jujuignore 5 '.bzr' => '.*/\\.bzr\\Z' :: Translated .jujuignore 6 '.tox' => '.*/\\.tox\\Z' :: Translated .jujuignore 8 '/build/' => '/build\\Z' :: Translated .jujuignore 9 '/revision' => '/revision\\Z' :: Translated .jujuignore 10 '/venv' => '/venv\\Z' :: Translated .jujuignore [12](https://github.com/canonical/istio-operators/actions/runs/9384901845/job/25841748232?pr=434#step:4:13) '.jujuignore' => '.*/\\.jujuignore\\Z' :: Translated .jujuignore 1 '/venv\n' => '/venv\\Z' :: Translated .jujuignore 2 '*.py[cod]\n' => '.*/[^/]*\\.py[cod]\\Z' :: Translated .jujuignore 3 '*.charm\n' => '.*/[^/]*\\.charm\\Z' :: Translated .jujuignore 4 '.tox\n' => '.*/\\.tox\\Z' :: Translated .jujuignore 5 '__pycache__\n' => '.*/__pycache__\\Z' :: Translated .jujuignore 1 '/staging-venv' => '/staging\\-venv\\Z' :: Collected charmlib dependencies: set() :: Building charm in '/root/parts/charm/install' :: Linking in generic paths :: Ignoring file because of rules: '.jujuignore' :: Creating the dispatch mechanism :: Creating the 'upgrade-charm' hook script pointing to dispatch :: Creating the 'install' hook script pointing to dispatch :: Creating the 'start' hook script pointing to dispatch :: Handling dependencies :: Current dependencies hash: 'd2b50d61430c50e878b34ed171619f2580b229af' :: Dependencies directory not found :: Installing dependencies :: Running external command ['python3', '-m', 'venv', '/root/parts/charm/build/staging-venv'] :: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', 'pip@https://files.pythonhosted.org/packages/ba/19/e63fb4e0d20e48bd2167bb7e857abc0e21679e24805ba921a224df8977c0/pip-23.2.1.tar.gz'] :: :: Requirement already satisfied: pip@ https://files.pythonhosted.org/packages/ba/19/e63fb4e0d20e48bd2167bb7e857abc0e21679e24805ba921a224df8977c0/pip-23.2.1.tar.gz from https://files.pythonhosted.org/packages/ba/19/e63fb4e0d20e48bd2167bb7e857abc0e21679e24805ba921a224df8977c0/pip-23.2.1.tar.gz in ./staging-venv/lib/python3.8/site-packages (20.0.2) :: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=anyio,attrs,certifi,charset-normalizer,exceptiongroup,h11,httpcore,httpx,idna,importlib-resources,jinja2,jsonschema,lightkube,lightkube-models,markupsafe,oci-image,ops,pip,pkgutil-resolve-name,pyrsistent,pyyaml,requests,serialized-data-interface,setuptools,sniffio,urllib3,websocket-client,wheel,zipp', '--requirement=requirements.txt', 'pip', 'setuptools', 'wheel'] :: :: Requirement already satisfied: pip in ./staging-venv/lib/python3.8/site-packages (20.0.2) :: :: Requirement already satisfied: setuptools in ./staging-venv/lib/python3.8/site-packages (44.0.0) :: :: Collecting wheel :: :: Downloading wheel-0.43.0.tar.gz (99 kB) :: :: Installing build dependencies: started :: :: Installing build dependencies: finished with status 'done' :: :: Getting requirements to build wheel: started :: :: Getting requirements to build wheel: finished with status 'done' :: :: Preparing wheel metadata: started :: :: Preparing wheel metadata: finished with status 'done' :: :: Collecting anyio==4.0.0 :: :: Downloading anyio-4.0.0.tar.gz (153 kB) :: :: Installing build dependencies: started :: :: Installing build dependencies: finished with status 'done' :: :: Getting requirements to build wheel: started :: :: Getting requirements to build wheel: finished with status 'done' :: :: Installing backend dependencies: started :: :: Installing backend dependencies: finished with status 'done' :: :: Preparing wheel metadata: started :: :: Preparing wheel metadata: finished with status 'done' :: :: Collecting attrs==23.1.0 :: :: Downloading attrs-23.1.0.tar.gz (212 kB) :: :: Installing build dependencies: started :: :: Installing build dependencies: finished with status 'done' :: :: Getting requirements to build wheel: started :: :: Getting requirements to build wheel: finished with status 'done' :: :: Preparing wheel metadata: started :: :: Preparing wheel metadata: finished with status 'done' :: :: Collecting certifi==2023.7.22 :: :: Downloading certifi-2023.7.22.tar.gz (159 kB) :: :: Collecting charset-normalizer==3.2.0 :: :: Downloading charset-normalizer-3.2.0.tar.gz (97 kB) :: :: Collecting exceptiongroup==1.1.3 :: :: Downloading exceptiongroup-1.1.3.tar.gz (23 kB) :: :: Installing build dependencies: started :: :: Installing build dependencies: finished with status 'done' :: :: Getting requirements to build wheel: started :: :: Getting requirements to build wheel: finished with status 'done' :: :: Preparing wheel metadata: started :: :: Preparing wheel metadata: finished with status 'done' :: :: Collecting h11==0.14.0 :: :: Downloading h11-0.14.0.tar.gz (100 kB) :: :: Installing build dependencies: started :: :: Installing build dependencies: finished with status 'done' :: :: Getting requirements to build wheel: started :: :: Getting requirements to build wheel: finished with status 'done' :: :: Preparing wheel metadata: started :: :: Preparing wheel metadata: finished with status 'done' :: :: Collecting httpcore==0.17.3 :: :: Downloading httpcore-0.17.3.tar.gz (62 kB) :: :: Collecting httpx==0.24.1 :: :: Downloading httpx-0.24.1.tar.gz (81 kB) :: :: Installing build dependencies: started :: :: Installing build dependencies: finished with status 'done' :: :: Getting requirements to build wheel: started :: :: Getting requirements to build wheel: finished with status 'done' :: :: Preparing wheel metadata: started :: :: Preparing wheel metadata: finished with status 'done' :: :: Collecting idna==3.4 :: :: Downloading idna-3.4.tar.gz (183 kB) :: :: Installing build dependencies: started :: :: Installing build dependencies: finished with status 'done' :: :: Getting requirements to build wheel: started :: :: Getting requirements to build wheel: finished with status 'done' :: :: Preparing wheel metadata: started :: :: Preparing wheel metadata: finished with status 'done' :: :: Collecting importlib-resources==6.0.1 :: :: Downloading importlib_resources-6.0.1.tar.gz (38 kB) :: :: Installing build dependencies: started :: :: Installing build dependencies: finished with status 'done' :: :: Getting requirements to build wheel: started :: :: Getting requirements to build wheel: finished with status 'done' :: :: Installing backend dependencies: started :: :: Installing backend dependencies: finished with status 'done' :: :: Preparing wheel metadata: started :: :: Preparing wheel metadata: finished with status 'done' :: :: Collecting jinja2==3.1.2 :: :: Downloading Jinja2-3.1.2.tar.gz (268 kB) :: :: ERROR: Command errored out with exit status 1: :: :: command: /root/parts/charm/build/staging-venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-j4xc92x7/jinja2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-j4xc92x7/jinja2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-j4xc92x7/jinja2/pip-egg-info :: :: cwd: /tmp/pip-install-j4xc92x7/jinja2/ :: :: Complete output (35 lines): :: :: Traceback (most recent call last): :: :: File "", line 1, in :: :: File "/tmp/pip-install-j4xc92x7/jinja2/setup.py", line 4, in :: :: setup( :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/__init__.py", line 145, in setup :: :: return distutils.core.setup(**attrs) :: :: File "/usr/lib/python3.8/distutils/core.py", line 121, in setup :: :: dist.parse_config_files() :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/dist.py", line 701, in parse_config_files :: :: parse_configuration(self, self.command_options, :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 121, in parse_configuration :: :: meta.parse() :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 426, in parse :: :: section_parser_method(section_options) :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 399, in parse_section :: :: self[name] = value :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 184, in __setitem__ :: :: value = parser(value) :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 515, in _parse_version :: :: version = self._parse_attr(value, self.package_dir) :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 349, in _parse_attr :: :: module = import_module(module_name) :: :: File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module :: :: return _bootstrap._gcd_import(name[level:], package, level) :: :: File "", line 1014, in _gcd_import :: :: File "", line 991, in _find_and_load :: :: File "", line 975, in _find_and_load_unlocked :: :: File "", line 671, in _load_unlocked :: :: File "", line 848, in exec_module :: :: File "", line 219, in _call_with_frames_removed :: :: File "/tmp/pip-install-j4xc92x7/jinja2/src/jinja2/__init__.py", line 8, in :: :: from .environment import Environment as Environment :: :: File "/tmp/pip-install-j4xc92x7/jinja2/src/jinja2/environment.py", line 14, in :: :: from markupsafe import Markup :: :: ModuleNotFoundError: No module named 'markupsafe' :: :: ---------------------------------------- :: :: ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. :: WARNING: Initial package installation failed. Falling back to older method, which may leave your charm in an un-runnable state. :: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--upgrade', '--no-binary', ':all:', 'wheel', 'setuptools', 'pip'] :: :: Collecting wheel :: :: Using cached wheel-0.43.0.tar.gz (99 kB) :: :: Installing build dependencies: started :: :: Installing build dependencies: finished with status 'done' :: :: Getting requirements to build wheel: started :: :: Getting requirements to build wheel: finished with status 'done' :: :: Preparing wheel metadata: started :: :: Preparing wheel metadata: finished with status 'done' :: :: Collecting setuptools :: :: Using cached setuptools-70.0.0.tar.gz (2.3 MB) :: :: Getting requirements to build wheel: started :: :: Getting requirements to build wheel: finished with status 'done' :: :: Installing backend dependencies: started :: :: Installing backend dependencies: finished with status 'done' :: :: Preparing wheel metadata: started :: :: Preparing wheel metadata: finished with status 'done' :: :: Collecting pip :: :: Downloading pip-24.0.tar.gz (2.1 MB) :: :: Installing build dependencies: started :: :: Installing build dependencies: finished with status 'done' :: :: Getting requirements to build wheel: started :: :: Getting requirements to build wheel: finished with status 'done' :: :: Preparing wheel metadata: started :: :: Preparing wheel metadata: finished with status 'done' :: :: Building wheels for collected packages: wheel, setuptools, pip :: :: Building wheel for wheel (PEP 517): started :: :: Building wheel for wheel (PEP 517): finished with status 'done' :: :: Created wheel for wheel: filename=wheel-0.43.0-py3-none-any.whl size=65775 sha256=55c570405f142630c6b9f72fe09d9b67cf1477fcf543ae5b8dcb1f5b7377da81 :: :: Stored in directory: /root/.cache/pip/wheels/d4/a3/b2/6faade3bb04cd720bb432a6c21ec7baa254280[13](https://github.com/canonical/istio-operators/actions/runs/9384901845/job/25841748232?pr=434#step:4:14)34a4d47a66 :: :: Building wheel for setuptools (PEP 517): started :: :: Building wheel for setuptools (PEP 517): finished with status 'done' :: :: Created wheel for setuptools: filename=setuptools-70.0.0-py3-none-any.whl size=863432 sha256=dc1c3e41e77a13dcc52aaf9051edcf91da0b06fceb9007f89d4a84e2e4294d7e :: :: Stored in directory: /root/.cache/pip/wheels/20/5f/b8/845eeb9e697798706931bd3ebc34b40c3bd3e3d6167407bb77 :: :: Building wheel for pip (PEP 517): started :: :: Building wheel for pip (PEP 517): finished with status 'done' :: :: Created wheel for pip: filename=pip-24.0-py3-none-any.whl size=2110223 sha256=741c1090d56683aca93c5c585d26dab507548a7e6afa75e59f9e76d1920a2359 :: :: Stored in directory: /root/.cache/pip/wheels/c1/d6/d4/2f7765d82b17d8c25425875b2dce5af9f22746668c3add50f6 :: :: Successfully built wheel setuptools pip :: :: Installing collected packages: wheel, setuptools, pip :: :: Attempting uninstall: setuptools :: :: Found existing installation: setuptools 44.0.0 :: :: Uninstalling setuptools-44.0.0: :: :: Successfully uninstalled setuptools-44.0.0 :: :: Rolling back uninstall of setuptools :: :: Moving to /root/parts/charm/build/staging-venv/bin/easy_install :: :: from /tmp/pip-uninstall-b6c35q6n/easy_install :: :: Moving to /root/parts/charm/build/staging-venv/bin/easy_install-3.8 :: :: from /tmp/pip-uninstall-b6c35q6n/easy_install-3.8 :: :: Moving to /root/parts/charm/build/staging-venv/lib/python3.8/site-packages/__pycache__/ :: :: from /root/parts/charm/build/staging-venv/lib/python3.8/site-packages/~_pycache__ :: :: Moving to /root/parts/charm/build/staging-venv/lib/python3.8/site-packages/easy_install.py :: :: from /tmp/pip-uninstall-_cmer5js/easy_install.py :: :: Moving to /root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools-44.0.0.dist-info/ :: :: from /root/parts/charm/build/staging-venv/lib/python3.8/site-packages/~etuptools-44.0.0.dist-info :: :: Moving to /root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/ :: :: from /root/parts/charm/build/staging-venv/lib/python3.8/site-packages/~etuptools :: :: ERROR: Exception: :: :: Traceback (most recent call last): :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 186, in _main :: :: status = self.run(options, args) :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 421, in run :: :: installed = install_given_reqs( :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/req/__init__.py", line 67, in install_given_reqs :: :: requirement.install( :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 809, in install :: :: install_wheel( :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/operations/install/wheel.py", line 606, in install_wheel :: :: unpack_file(wheel_path, unpacked_dir.path) :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/utils/unpacking.py", line 249, in unpack_file :: :: unzip_file( :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/utils/unpacking.py", line 139, in unzip_file :: :: shutil.copyfileobj(fp, destfp) :: :: File "/usr/lib/python3.8/shutil.py", line 205, in copyfileobj :: :: buf = fsrc_read(length) :: :: File "/usr/lib/python3.8/zipfile.py", line 940, in read :: :: data = self._read1(n) :: :: File "/usr/lib/python3.8/zipfile.py", line 1008, in _read1 :: :: data += self._read2(n - len(data)) :: :: File "/usr/lib/python3.8/zipfile.py", line 1043, in _read2 :: :: raise EOFError :: :: EOFError :: Traceback (most recent call last): :: File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 260, in _install_dependencies :: _process_run( :: File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 408, in _process_run :: raise RuntimeError(f"Subprocess command {cmd} execution failed with retcode {retcode}") :: RuntimeError: Subprocess command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=anyio,attrs,certifi,charset-normalizer,exceptiongroup,h11,httpcore,httpx,idna,importlib-resources,jinja2,jsonschema,lightkube,lightkube-models,markupsafe,oci-image,ops,pip,pkgutil-resolve-name,pyrsistent,pyyaml,requests,serialized-data-interface,setuptools,sniffio,urllib3,websocket-client,wheel,zipp', '--requirement=requirements.txt', 'pip', 'setuptools', 'wheel'] execution failed with retcode 1 :: :: During handling of the above exception, another exception occurred: :: :: Traceback (most recent call last): :: File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 477, in :: main() :: File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 472, in main :: builder.build_charm() :: File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 101, in build_charm :: self.handle_dependencies() :: File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 354, in handle_dependencies :: self._install_dependencies(staging_venv_dir) :: File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/instrum.py", line 152, in _f :: return func(*args, **kwargs) :: File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 283, in _install_dependencies :: _process_run(cmd) :: File "/snap/charmcraft/3454/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 408, in _process_run :: raise RuntimeError(f"Subprocess command {cmd} execution failed with retcode {retcode}") :: RuntimeError: Subprocess command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--upgrade', '--no-binary', ':all:', 'wheel', 'setuptools', 'pip'] execution failed with retcode 2 Parts processing error: Failed to run the build script for part 'charm'. Failed to build charm for bases index '0'. Full execution log: '/home/runner/.local/state/charmcraft/log/charmcraft-20240605-132615.576[14](https://github.com/canonical/istio-operators/actions/runs/9384901845/job/25841748232?pr=434#step:4:15)2.log' =========================== short test summary info ============================ FAILED tests/test_bundle_tls_provider.py::test_build_and_deploy_istio_charms =================== 1 failed, 1 xfailed in [15](https://github.com/canonical/istio-operators/actions/runs/9384901845/job/25841748232?pr=434#step:4:16)7.93s (0:02:37) =================== integration-tls-provider: exit 1 (158.71 seconds) /home/runner/work/istio-operators/istio-operators> pytest --show-capture=no --log-cli-level=INFO -vvs --tb=native --model test-istio tests/test_bundle_tls_provider.py pid=[16](https://github.com/canonical/istio-operators/actions/runs/9384901845/job/25841748232?pr=434#step:4:17)652 integration-tls-provider: FAIL code 1 ([17](https://github.com/canonical/istio-operators/actions/runs/9384901845/job/25841748232?pr=434#step:4:18)4.96=setup[16.25]+cmd[158.71] seconds) evaluation failed :( (175.03 seconds) ```
DnPlas commented 3 months ago

Moving this issue to canonical/bundle-kubeflow as it is happening for more than one charm.

syncronize-issues-to-jira[bot] commented 3 months ago

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-6081.

This message was autogenerated

orfeas-k commented 2 months ago

Debugging

orfeas-k commented 2 months ago

Workaround

The issue was worked around in #531 by using charmcraft from latest/candidate in integration tests and backporting changes that free disk space in runner #519 and updating cert_handler library to v1 #517.

Cause

Doing a little more debugging, it looks like the issue could be caused because we are building two charms at the same time, since:

Solution

In the future, we could hit this again when charmcraft from latest/edge is promoted. If this happens, we should allocate time to work on a larger effort to refactor our CI as described e.g. in #766 or https://github.com/canonical/istio-operators/pull/412, which would probably eliminate this issue as well.