search

canonical / kfp-operators

Kubeflow Pipelines Operators
Apache License 2.0
2 stars 12 forks source link

Error when building charms simultaneously: BadZipFile error #564

Closed mvlassis closed 2 months ago

mvlassis commented 2 months ago

Bug Description

This issue is encountered while building charms simultaneously using tox. This unfortunately blocks our CI, as seen in these run logs. It seems that this is related to charmcraft, as this issue was first documented in the charmcraft repository.

To Reproduce

I have reproduced the error message locally.

Environment

Charmcraft version: latest/candidate which is currently 4.2.0 Juju version: 3.4.5 Python version: 3.8.19

Relevant Log Output

=========================== short test summary info ============================
FAILED tests/integration/test_kfp_functional_v2.py::test_build_and_deploy - RuntimeError: Failed to build charm /home/runner/work/kfp-operators/kfp-operators/charms/kfp-schedwf:
Launching managed ubuntu 20.04 instance...
Creating new instance from remote
Creating new base instance from remote
Creating new instance from base instance
Starting instance
Initialising lifecycle
Installing build-packages
Pulling charm
Building charm
:: + env -i LANG=C.UTF-8 LC_ALL=C.UTF-8 CRYPTOGRAPHY_OPENSSL_NO_LEGACY=true PATH=/snap/charmcraft/4684/libexec/charmcraft:/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin SNAP=/snap/charmcraft/4684 SNAP_ARCH=amd64 SNAP_NAME=charmcraft SNAP_VERSION=3.2.0 /snap/charmcraft/4684/bin/python3 -u -I /snap/charmcraft/4684/lib/python3.10/site-packages/charmcraft/charm_builder.py --builddir /root/parts/charm/build --installdir /root/parts/charm/install --entrypoint /root/parts/charm/build/src/charm.py -p wheel -p setuptools -p pip -r requirements.txt
:: Starting charm builder
:: Translated .jujuignore 2 '.git' => '.*/\\.git\\Z'
:: Translated .jujuignore 3 '.svn' => '.*/\\.svn\\Z'
:: Translated .jujuignore 4 '.hg' => '.*/\\.hg\\Z'
:: Translated .jujuignore 5 '.bzr' => '.*/\\.bzr\\Z'
:: Translated .jujuignore 6 '.tox' => '.*/\\.tox\\Z'
:: Translated .jujuignore 8 '/build/' => '/build\\Z'
:: Translated .jujuignore 9 '/revision' => '/revision\\Z'
:: Translated .jujuignore 10 '/venv' => '/venv\\Z'
:: Translated .jujuignore 12 '.jujuignore' => '.*/\\.jujuignore\\Z'
:: Translated .jujuignore 1 '/staging-venv' => '/staging\\-venv\\Z'
:: Collected charmlib dependencies: {'cosl'}
:: Building charm in '/root/parts/charm/install'
:: Linking in generic paths
:: Creating the dispatch mechanism
:: Creating the 'start' hook script pointing to dispatch
:: Creating the 'upgrade-charm' hook script pointing to dispatch
:: Creating the 'install' hook script pointing to dispatch
:: Handling dependencies
:: Current dependencies hash: 'de101206b8507f64c21bbd7959ecc6ff5dc80f35'
:: Dependencies directory not found
:: Installing dependencies
:: Running external command ['python3', '-m', 'venv', '/root/parts/charm/build/staging-venv']
:: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', 'pip@https://files.pythonhosted.org/packages/c0/d0/9641dc7b05877874c6418f8034ddefc809495e65caa14d38c7551cd114bb/pip-24.1.1.tar.gz']
::    :: Requirement already satisfied: pip@ https://files.pythonhosted.org/packages/c0/d0/9641dc7b05877874c6418f8034ddefc809495e65caa14d38c7551cd114bb/pip-24.1.1.tar.gz from https://files.pythonhosted.org/packages/c0/d0/9641dc7b05877874c6418f8034ddefc809495e65caa14d38c7551cd114bb/pip-24.1.1.tar.gz in ./staging-venv/lib/python3.8/site-packages (20.0.2)
:: Installing Python pre-dependencies from source.
:: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=:all:', 'wheel', 'setuptools', 'pip']
::    :: Collecting wheel
::    ::   Using cached wheel-0.44.0.tar.gz (100 kB)
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: finished with status 'done'
::    ::   Getting requirements to build wheel: started
::    ::   Getting requirements to build wheel: finished with status 'done'
::    ::     Preparing wheel metadata: started
::    ::     Preparing wheel metadata: finished with status 'done'
::    :: Requirement already satisfied: setuptools in ./staging-venv/lib/python3.8/site-packages (44.0.0)
::    :: Requirement already satisfied: pip in ./staging-venv/lib/python3.8/site-packages (20.0.2)
::    :: Building wheels for collected packages: wheel
::    ::   Building wheel for wheel (PEP 517): started
::    ::   Building wheel for wheel (PEP 517): finished with status 'done'
::    ::   Created wheel for wheel: filename=wheel-0.44.0-py3-none-any.whl size=67059 sha256=2376a90c98cc337d18623527a97c31797bd02bad0033d41547043a1cbfbe448f
::    ::   Stored in directory: /root/.cache/pip/wheels/e2/ec/77/3b86b29474a5ff9fe2c368abefa00b6cc86dbe6b7db87ffe1e
::    :: Successfully built wheel
::    :: Installing collected packages: wheel
::    :: Successfully installed wheel-0.44.0
:: Installing packages from requirements files and charm lib dependencies.
:: Running external command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=:all:', '--requirement=requirements.txt']
::    :: Collecting anyio==3.7.1
::    ::   Using cached anyio-3.7.1.tar.gz (142 kB)
::    ::   Installing build dependencies: started
::    ::   Installing build dependencies: still running...
::    ::   Installing build dependencies: finished with status 'error'
::    ::   ERROR: Command errored out with exit status 2:
::    ::    command: /root/parts/charm/build/staging-venv/bin/python3 /root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-f651i_hx/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools >= 64' 'setuptools_scm >= 6.4'
::    ::        cwd: None
::    ::   Complete output (80 lines):
::    ::   Collecting setuptools>=64
::    ::     Using cached setuptools-74.0.0.tar.gz (1.4 MB)
::    ::     Getting requirements to build wheel: started
::    ::     Getting requirements to build wheel: finished with status 'done'
::    ::       Preparing wheel metadata: started
::    ::       Preparing wheel metadata: finished with status 'done'
::    ::   Collecting setuptools_scm>=6.4
::    ::     Using cached setuptools_scm-8.1.0.tar.gz (76 kB)
::    ::     Installing build dependencies: started
::    ::     Installing build dependencies: finished with status 'done'
::    ::     Getting requirements to build wheel: started
::    ::     Getting requirements to build wheel: finished with status 'done'
::    ::       Preparing wheel metadata: started
::    ::       Preparing wheel metadata: finished with status 'done'
::    ::   Collecting tomli>=1; python_version < "3.11"
::    ::     Using cached tomli-2.0.1.tar.gz (15 kB)
::    ::     Installing build dependencies: started
::    ::     Installing build dependencies: finished with status 'done'
::    ::     Getting requirements to build wheel: started
::    ::     Getting requirements to build wheel: finished with status 'done'
::    ::       Preparing wheel metadata: started
::    ::       Preparing wheel metadata: finished with status 'done'
::    ::   Collecting packaging>=20
::    ::     Using cached packaging-24.1.tar.gz (148 kB)
::    ::     Installing build dependencies: started
::    ::     Installing build dependencies: finished with status 'done'
::    ::     Getting requirements to build wheel: started
::    ::     Getting requirements to build wheel: finished with status 'done'
::    ::       Preparing wheel metadata: started
::    ::       Preparing wheel metadata: finished with status 'done'
::    ::   Collecting typing-extensions; python_version < "3.10"
::    ::     Using cached typing_extensions-4.12.2.tar.gz (85 kB)
::    ::     Installing build dependencies: started
::    ::     Installing build dependencies: finished with status 'done'
::    ::     Getting requirements to build wheel: started
::    ::     Getting requirements to build wheel: finished with status 'done'
::    ::       Preparing wheel metadata: started
::    ::       Preparing wheel metadata: finished with status 'done'
::    ::   Building wheels for collected packages: setuptools, setuptools-scm, tomli, packaging, typing-extensions
::    ::     Building wheel for setuptools (PEP 517): started
::    ::     Building wheel for setuptools (PEP 517): finished with status 'done'
::    ::     Created wheel for setuptools: filename=setuptools-74.0.0-py3-none-any.whl size=1301729 sha256=82c1ebfdbab26ee4fe501bcbee4cf3532e713fdca566fe94178117f09a171a68
::    ::     Stored in directory: /root/.cache/pip/wheels/1d/17/f1/d2a056eb3ee4fd455893c3fa5b6e5a617d67682a395b82936a
::    ::     Building wheel for setuptools-scm (PEP 517): started
::    ::     Building wheel for setuptools-scm (PEP 517): finished with status 'done'
::    ::     Created wheel for setuptools-scm: filename=setuptools_scm-8.1.0-py3-none-any.whl size=43666 sha256=0f69319b0765d816dbf5c3feb9ebe8914b06b9070e94cbc58cd101389acda548
::    ::     Stored in directory: /root/.cache/pip/wheels/4f/0c/2a/dd9ab1cfd42b47ba620697db889c01ea8a95fb7d7922a1538e
::    ::     Building wheel for tomli (PEP 517): started
::    ::     Building wheel for tomli (PEP 517): finished with status 'done'
::    ::     Created wheel for tomli: filename=tomli-2.0.1-py3-none-any.whl size=12758 sha256=b57d53172b3432857125ceaa4b9b2d28e4ae0510c95c5cf031955ada144ba3ff
::    ::     Stored in directory: /root/.cache/pip/wheels/bb/b6/90/165ed2d4e3933c7567df3ca84de8db1cf47278a1673e8c476c
::    ::     Building wheel for packaging (PEP 517): started
::    ::     Building wheel for packaging (PEP 517): finished with status 'done'
::    ::     Created wheel for packaging: filename=packaging-24.1-py3-none-any.whl size=53985 sha256=270c5ae91ef4184b318179b819b2e4420e8f41246d05334899bc9dfc30c80cac
::    ::     Stored in directory: /root/.cache/pip/wheels/69/d8/fd/03595f42fbfd16f9988c5754c8c1409f596819d22b19c3986f
::    ::     Building wheel for typing-extensions (PEP 517): started
::    ::     Building wheel for typing-extensions (PEP 517): finished with status 'done'
::    ::     Created wheel for typing-extensions: filename=typing_extensions-4.12.2-py3-none-any.whl size=37438 sha256=04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d
::    ::     Stored in directory: /root/.cache/pip/wheels/9d/46/a1/b5cb19abe2676081c6f4cf561590531a88becbbe7e04205a63
::    ::   Successfully built setuptools setuptools-scm tomli packaging typing-extensions
::    ::   Installing collected packages: setuptools, tomli, packaging, typing-extensions, setuptools-scm
::    ::   ERROR: Exception:
::    ::   Traceback (most recent call last):
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
::    ::       status = self.run(options, args)
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 421, in run
::    ::       installed = install_given_reqs(
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/req/__init__.py", line 67, in install_given_reqs
::    ::       requirement.install(
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 809, in install
::    ::       install_wheel(
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/operations/install/wheel.py", line 606, in install_wheel
::    ::       unpack_file(wheel_path, unpacked_dir.path)
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/utils/unpacking.py", line 249, in unpack_file
::    ::       unzip_file(
::    ::     File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip/_internal/utils/unpacking.py", line 136, in unzip_file
::    ::       fp = zip.open(name)
::    ::     File "/usr/lib/python3.8/zipfile.py", line 1564, in open
::    ::       raise BadZipFile(
::    ::   zipfile.BadZipFile: File name in directory 'setuptools/_vendor/importlib_resources/tests/namespacedata01/utf-8.file' and header b'setuptools/_vendor/importlib_re' differ.
::    ::   ----------------------------------------
::    :: ERROR: Command errored out with exit status 2: /root/parts/charm/build/staging-venv/bin/python3 /root/parts/charm/build/staging-venv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-f651i_hx/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'setuptools >= 64' 'setuptools_scm >= 6.4' Check the logs for full command output.
:: Traceback (most recent call last):
::   File "/snap/charmcraft/4684/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 482, in <module>
::     main()
::   File "/snap/charmcraft/4684/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 477, in main
::     builder.build_charm()
::   File "/snap/charmcraft/4684/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 93, in build_charm
::     self.handle_dependencies()
::   File "/snap/charmcraft/4684/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 353, in handle_dependencies
::     self._install_dependencies(staging_venv_dir)
::   File "/snap/charmcraft/4684/lib/python3.10/site-packages/charmcraft/instrum.py", line 152, in _f
::     return func(*args, **kwargs)
::   File "/snap/charmcraft/4684/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 283, in _install_dependencies
::     _process_run(
::   File "/snap/charmcraft/4684/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 413, in _process_run
::     raise RuntimeError(f"Subprocess command {cmd} execution failed with retcode {retcode}")
:: RuntimeError: Subprocess command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=:all:', '--requirement=requirements.txt'] execution failed with retcode 1
Failed to run the build script for part 'charm'.
Recommended resolution: Check the build output and verify the project can work with the 'charm' plugin.
Failed to execute charmcraft in instance.
Full execution log: '/home/runner/.local/state/charmcraft/log/charmcraft-20240830-084229.848676.log'
ERROR tests/integration/test_kfp_functional_v2.py::test_upload_pipeline - httpx.HTTPStatusError: Client error '404 Not Found' for url 'https://10.1.0.18:16443/apis/kubeflow.org/v1/profiles/kubeflow-user-example-com?fieldManager=kfp-operators'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404
ERROR tests/integration/test_kfp_functional_v2.py::test_create_and_monitor_run - httpx.HTTPStatusError: Client error '404 Not Found' for url 'https://10.1.0.18:16443/apis/kubeflow.org/v1/profiles/kubeflow-user-example-com?fieldManager=kfp-operators'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404
ERROR tests/integration/test_kfp_functional_v2.py::test_create_and_monitor_recurring_run - httpx.HTTPStatusError: Client error '404 Not Found' for url 'https://10.1.0.18:16443/apis/kubeflow.org/v1/profiles/kubeflow-user-example-com?fieldManager=kfp-operators'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404
======== 1 failed, 2 xfailed, 1 warning, 3 errors in 334.47s (0:05:34) =========
bundle-integration-v2: exit 1 (339.56 seconds) /home/runner/work/kfp-operators/kfp-operators> pytest -vv --tb=native -s --model kubeflow --bundle=./tests/integration/bundles/kfp_latest_edge.yaml.j2 /home/runner/work/kfp-operators/kfp-operators/tests/integration/test_kfp_functional_v2.py pid=145848
  bundle-integration-v2: FAIL code 1 (362.19=setup[22.63]+cmd[339.56] seconds)
  evaluation failed :( (362.32 seconds)
Error: Process completed with exit code 1.

Additional Context

No response

syncronize-issues-to-jira[bot] commented 2 months ago

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-6193.

This message was autogenerated

mvlassis commented 2 months ago

Closing as I created this issue to track the error in all of our repositories.