search

borgbackup / borg

Deduplicating archiver with compression and authenticated encryption.
https://www.borgbackup.org/
Other
10.79k stars 734 forks source link

windows: github CI broken #8264

Open ThomasWaldmann opened 2 weeks ago

ThomasWaldmann commented 2 weeks ago

https://github.com/borgbackup/borg/actions/runs/9766803308/job/26960577084?pr=8263

ThomasWaldmann commented 2 weeks ago 
Run ./scripts/msys2-build
Obtaining file:///D://a/borg/borg
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Checking if build backend supports build_editable: started
  Checking if build backend supports build_editable: finished with status 'done'
  Getting requirements to build editable: started
  Getting requirements to build editable: finished with status 'done'
  Preparing editable metadata (pyproject.toml): started
  Preparing editable metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: msgpack<=1.1.0,>=1.0.3 in d:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages (from borgbackup==1.4.0rc2.dev13+g76be26d4.d20240702) (1.0.6)
Requirement already satisfied: packaging in d:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages (from borgbackup==1.4.0rc2.dev13+g76be26d4.d20240702) (24.1)
Building wheels for collected packages: borgbackup
  Building editable for borgbackup (pyproject.toml): started
  Building editable for borgbackup (pyproject.toml): still running...
  Building editable for borgbackup (pyproject.toml): finished with status 'done'
  Created wheel for borgbackup: filename=borgbackup-1.4.0rc2.dev13+g76be26d4.d20240702-0.editable-cp311-cp311-mingw_x86_64_ucrt.whl size=7625 sha256=d72c0ff4aa6b9bf80bc940bdf50da0a4b92bc62081ca0697e91f48b5774a3cc8
  Stored in directory: D:/a/_temp/msys64/tmp/pip-ephem-wheel-cache-mczy8ud2/wheels/bb/6d/6e/fdef5aca9f58a65a983afbc55f7dae75316edcc2fbf171af32
Successfully built borgbackup
Installing collected packages: borgbackup
Successfully installed borgbackup-1.4.0rc2.dev13+g76be26d4.d20240702
171 INFO: PyInstaller: 6.3.0
171 INFO: Python: 3.11.9
171 INFO: Platform: Windows-10-10.0.20348-SP0
171 INFO: Extending PYTHONPATH with paths
['D:/a/borg/borg/src', 'D:/a/borg/borg']
296 INFO: Appending 'datas' from .spec
296 INFO: checking Analysis
296 INFO: Building Analysis because Analysis-00.toc is non existent
296 INFO: Initializing module dependency graph...
296 INFO: Caching module graph hooks...
312 INFO: Analyzing base_library.zip ...
1078 INFO: Loading module hook 'hook-heapq.py' from 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks'...
1328 INFO: Loading module hook 'hook-encodings.py' from 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks'...
2611 INFO: Loading module hook 'hook-pickle.py' from 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks'...
3705 INFO: Caching module dependency graph...
3814 INFO: Running Analysis Analysis-00.toc
3814 INFO: Looking for Python shared library...
3814 INFO: Using Python shared library: D:/a/_temp/msys64/ucrt64/bin/libpython3.11.dll
3814 INFO: Analyzing D:/a/borg/borg/src/borg/__main__.py
3830 INFO: Loading module hook 'hook-packaging.py' from 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks'...
4205 INFO: Loading module hook 'hook-platform.py' from 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks'...
4783 INFO: Loading module hook 'hook-difflib.py' from 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks'...
4955 INFO: Loading module hook 'hook-multiprocessing.util.py' from 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks'...
5049 INFO: Loading module hook 'hook-xml.py' from 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks'...
5565 INFO: Loading module hook 'hook-sysconfig.py' from 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks'...
5722 INFO: Processing module hooks...
5800 INFO: Loading module hook 'hook-pkg_resources.py' from 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks'...
6988 INFO: Performing binary vs. data reclassification (3 entries)
6988 INFO: Looking for ctypes DLLs
7019 INFO: Analyzing run-time hooks ...
7019 INFO: Including run-time hook 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_inspect.py'
7019 INFO: Including run-time hook 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py'
7019 INFO: Processing pre-find module path hook _pyi_rth_utils from 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks/pre_find_module_path/hook-_pyi_rth_utils.py'.
7019 INFO: Loading module hook 'hook-_pyi_rth_utils.py' from 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks'...
7019 INFO: Including run-time hook 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgres.py'
7019 INFO: Including run-time hook 'D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
7036 INFO: Looking for dynamic libraries
7333 INFO: Extra DLL search directories (AddDllDirectory): []
7333 INFO: Extra DLL search directories (PATH): []
7739 INFO: Warnings written to D:/a/borg/borg/build/borg.exe/warn-borg.exe.txt
7786 INFO: Graph cross-reference written to D:/a/borg/borg/build/borg.exe/xref-borg.exe.html
7801 INFO: checking PYZ
7801 INFO: Building PYZ because PYZ-00.toc is non existent
7801 INFO: Building PYZ (ZlibArchive) D:/a/borg/borg/build/borg.exe/PYZ-00.pyz
8396 INFO: Building PYZ (ZlibArchive) D:/a/borg/borg/build/borg.exe/PYZ-00.pyz completed successfully.
8411 INFO: checking PKG
8411 INFO: Building PKG because PKG-00.toc is non existent
8411 INFO: Building PKG (CArchive) borg.pkg
10631 INFO: Building PKG (CArchive) borg.pkg completed successfully.
10631 INFO: Bootloader D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/bootloader/Windows-64bit-intel/run.exe
10631 INFO: checking EXE
10631 INFO: Building EXE because EXE-00.toc is non existent
10631 INFO: Building EXE from EXE-00.toc
10631 INFO: Copying bootloader EXE to D:/a/borg/borg/dist/borg.exe
10631 INFO: Copying icon to EXE
10631 INFO: Copying 0 resources to EXE
10631 INFO: Embedding manifest in EXE
10631 INFO: Appending PKG archive to EXE
10647 INFO: Fixing EXE headers
10709 INFO: Building EXE from EXE-00.toc completed successfully.
10725 INFO: checking PKG
10725 INFO: Building PKG because PKG-01.toc is non existent
10725 INFO: Building PKG (CArchive) borg.pkg
10741 INFO: Building PKG (CArchive) borg.pkg completed successfully.
10741 INFO: Bootloader D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/PyInstaller/bootloader/Windows-64bit-intel/run.exe
10741 INFO: checking EXE
10741 INFO: Building EXE because EXE-01.toc is non existent
10741 INFO: Building EXE from EXE-01.toc
10741 INFO: Copying bootloader EXE to D:/a/borg/borg/build/borg.exe/borg.exe
10741 INFO: Copying icon to EXE
10741 INFO: Copying 0 resources to EXE
10741 INFO: Embedding manifest in EXE
10741 INFO: Appending PKG archive to EXE
10741 INFO: Fixing EXE headers
10819 INFO: Building EXE from EXE-01.toc completed successfully.
10819 INFO: checking COLLECT
10819 INFO: Building COLLECT because COLLECT-00.toc is non existent
10819 INFO: Building COLLECT COLLECT-00.toc
10850 INFO: Building COLLECT COLLECT-00.toc completed successfully.
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
< Traceback (most recent call last):
  - Cython>=3.0.3
  - pkgconfig
  - setuptools>=64
  - setuptools_scm>=8
  - wheel
> D:/a/_temp/msys64/ucrt64/bin/python.exe -m pip --python
  D:/a/_temp/msys64/tmp/build-env-jm8rgbnn/bin/python.exe install --use-pep517
  --no-warn-script-location --no-compile -r D:/a/_temp/msys64/tmp/build-reqs-
  mhinqb17.txt
<   File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/pip/__pip-
  runner__.py", line 50, in <module>
<     runpy.run_module("pip", run_name="__main__", alter_sys=True)
<   File "<frozen runpy>", line 226, in run_module
<   File "<frozen runpy>", line 98, in _run_module_code
<   File "<frozen runpy>", line 88, in _run_code
<   File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-
  packages/pip/__main__.py", line 24, in <module>
<     sys.exit(_main())
<              ^^^^^^^
<   File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-
  packages/pip/_internal/cli/main.py", line 77, in main
<     command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
<               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<   File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-
  packages/pip/_internal/commands/__init__.py", line 114, in create_command
<     module = importlib.import_module(module_path)
<              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<   File "D:/a/_temp/msys64/ucrt64/lib/python3.11/importlib/__init__.py", line
  126, in import_module
<     return _bootstrap._gcd_import(name[level:], package, level)
<            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
<   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
<   File "<frozen importlib._bootstrap>", line 1147, in
  _find_and_load_unlocked
<   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
<   File "<frozen importlib._bootstrap_external>", line 944, in exec_module
<   File "<frozen importlib._bootstrap>", line 241, in
  _call_with_frames_removed
<   File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-
  packages/pip/_internal/commands/install.py", line 15, in <module>
<     from pip._internal.cli.req_command import (
<   File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-
  packages/pip/_internal/cli/req_command.py", line 21, in <module>
<     from pip._internal.index.package_finder import PackageFinder
<   File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-
  packages/pip/_internal/index/package_finder.py", line 30, in <module>
<     from pip._internal.req import InstallRequirement
<   File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-
  packages/pip/_internal/req/__init__.py", line 8, in <module>
<     from .req_install import InstallRequirement
<   File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-
  packages/pip/_internal/req/req_install.py", line 40, in <module>
<     from pip._internal.operations.install.wheel import install_wheel
<   File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-
  packages/pip/_internal/operations/install/wheel.py", line 39, in <module>
<     from pip._vendor.distlib.scripts import ScriptMaker
< ModuleNotFoundError: No module named 'pip._vendor.distlib'

Traceback (most recent call last):
  File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/build/__main__.py", line 178, in _handle_build_error
    yield
  File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/build/__main__.py", line 429, in main
    built = build_call(
            ^^^^^^^^^^^
  File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/build/__main__.py", line 268, in build_package_via_sdist
    sdist = _build(isolation, srcdir, outdir, 'sdist', config_settings, skip_dependency_check, installer)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/build/__main__.py", line 170, in _build
    return _build_in_isolated_env(srcdir, outdir, distribution, config_settings, installer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/build/__main__.py", line 135, in _build_in_isolated_env
    env.install(builder.build_system_requires)
  File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/build/env.py", line 136, in install
    self._env_backend.install_requirements(requirements)
  File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/build/env.py", line 265, in install_requirements
    run_subprocess(cmd)
  File "D:/a/_temp/msys64/ucrt64/lib/python3.11/site-packages/build/_ctx.py", line 71, in run_subprocess
    subprocess.run(cmd, capture_output=True, check=True, env=env)
  File "D:/a/_temp/msys64/ucrt64/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['D:/a/_temp/msys64/ucrt64/bin/python.exe', '-m', 'pip', '--python', 'D:/a/_temp/msys64/tmp/build-env-jm8rgbnn/bin/python.exe', 'install', '--use-pep517', '--no-warn-script-location', '--no-compile', '-r', 'D:/a/_temp/msys64/tmp/build-reqs-mhinqb17.txt']' returned non-zero exit status 1.

ERROR Command '['D:/a/_temp/msys64/ucrt64/bin/python.exe', '-m', 'pip', '--python', 'D:/a/_temp/msys64/tmp/build-env-jm8rgbnn/bin/python.exe', 'install', '--use-pep517', '--no-warn-script-location', '--no-compile', '-r', 'D:/a/_temp/msys64/tmp/build-reqs-mhinqb17.txt']' returned non-zero exit status 1.
Error: Process completed with exit code 1.
Aztorius commented 2 weeks ago

Some jobs have been using pyinstaller 6.7.0 and 6.8.0 lately which are working fine. But then later jobs where using 6.3.0 which seems to fail now. Maybe there is some cache issues in github ci. Possible solutions : upgrade pyinstaller or remove cache for msys2.

ThomasWaldmann commented 1 week ago

Upgrading pyinstaller (windows CI used 6.3.0):

So, it doesn't look like the failure is related to the pyi version.

< ModuleNotFoundError: No module named 'pip._vendor.distlib'

Rather something is broken inside pip.

ThomasWaldmann commented 1 week ago

Interestingly, 1.2-maint branch is not broken, while 1.4-maint and master are.

Maybe related to the more modern setup (pyproject.toml etc.) in 1.4-maint and master?

For now, I disabled the windows msys2 jobs in 1.4-maint and master until this gets fixed.