pip_install of 'cairocffi' fails

[dunkmann00]

Hello, I was trying to include the library cairosvg into my project using exe.add_python_resources(exe.pip_install(["cairosvg"])) but was met with an error. cairosvg has a dependency on cairocffi and this is the library that caused the error when trying to install.

To confirm the problem was just with cairocffi I changed my bzl script to try to specifically install only cairocffi. Here is the output from trying to build a binary with PyOxidizer:

--- Building binary with PyOxidizer. ---
/Users/georgewaters/Documents/Developer/Python/Monopoly-Probabilities/venv/bin/pyoxidizer build --release --var PYOXIDIZER_BUILD_DIR pyoxidizer-build
resolving 1 targets
resolving target install
resolving target exe
target Python distribution for x86_64-apple-darwin resolves to: https://github.com/indygreg/python-build-standalone/releases/download/20220528/cpython-3.10.4%2B20220528-x86_64-apple-darwin-pgo-full.tar.zst (sha256=823a456d3d6ebbf8d81604dd875a8091f57550be18959cb2a97fc8b25e2f2d87)
reusing target Python distribution for host execution
installing modified distutils to /var/folders/06/4dkcltv94pz6jp2fl73c646w0000gn/T/pyoxidizer-pip-installmBLV4o/packages
modifying distutils/_msvccompiler.py for oxidation
modifying distutils/command/build_ext.py for oxidation
modifying distutils/unixccompiler.py for oxidation
pip installing to /var/folders/06/4dkcltv94pz6jp2fl73c646w0000gn/T/pyoxidizer-pip-installmBLV4o/install
Processing /Users/georgewaters/Documents/Developer/Python/Monopoly-Probabilities
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Installing backend dependencies: started
  Installing backend dependencies: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: monopoly-probabilities
  Building wheel for monopoly-probabilities (pyproject.toml): started
  Building wheel for monopoly-probabilities (pyproject.toml): finished with status 'done'
  Created wheel for monopoly-probabilities: filename=monopoly_probabilities-0.1.0-cp310-cp310-macosx_12_0_x86_64.whl size=109675 sha256=55f1ca14dd6ee5d1d6f64f8ab26f6d0ff5422ed29566a3d10ed7482bf0183460
  Stored in directory: /Users/georgewaters/Library/Caches/pip/wheels/b0/17/2f/fc165bb28d941e39027dfaf7eede213058cf4b3502fdf01093
Successfully built monopoly-probabilities
Installing collected packages: monopoly-probabilities
Successfully installed monopoly-probabilities-0.1.0
installing modified distutils to /var/folders/06/4dkcltv94pz6jp2fl73c646w0000gn/T/pyoxidizer-pip-installHSyY8B/packages
modifying distutils/_msvccompiler.py for oxidation
modifying distutils/command/build_ext.py for oxidation
modifying distutils/unixccompiler.py for oxidation
pip installing to /var/folders/06/4dkcltv94pz6jp2fl73c646w0000gn/T/pyoxidizer-pip-installHSyY8B/install
Collecting cairocffi
  Using cached cairocffi-1.4.0.tar.gz (69 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting cffi>=1.1.0
  Using cached cffi-1.15.1-cp310-cp310-macosx_10_9_x86_64.whl (179 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Using legacy 'setup.py install' for cairocffi, since package 'wheel' is not installed.
Installing collected packages: pycparser, cffi, cairocffi
  Running setup.py install for cairocffi: started
  Running setup.py install for cairocffi: finished with status 'error'
  error: subprocess-exited-with-error

  × Running setup.py install for cairocffi did not run successfully.
  │ exit code: 1
  ╰─> [31 lines of output]
      /Users/georgewaters/Library/Caches/pyoxidizer/python_distributions/python.823a456d3d6e/python/install/lib/python3.10/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
        warnings.warn(
      WARNING: The wheel package is not available.
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/private/var/folders/06/4dkcltv94pz6jp2fl73c646w0000gn/T/pip-install-5ccjc5qv/cairocffi_a98e862e506d4c96a615da379f0de407/setup.py", line 10, in <module>
          setup(
        File "/Users/georgewaters/Library/Caches/pyoxidizer/python_distributions/python.823a456d3d6e/python/install/lib/python3.10/site-packages/setuptools/__init__.py", line 86, in setup
          _install_setup_requires(attrs)
        File "/Users/georgewaters/Library/Caches/pyoxidizer/python_distributions/python.823a456d3d6e/python/install/lib/python3.10/site-packages/setuptools/__init__.py", line 80, in _install_setup_requires
          dist.fetch_build_eggs(dist.setup_requires)
        File "/Users/georgewaters/Library/Caches/pyoxidizer/python_distributions/python.823a456d3d6e/python/install/lib/python3.10/site-packages/setuptools/dist.py", line 876, in fetch_build_eggs
          resolved_dists = pkg_resources.working_set.resolve(
        File "/Users/georgewaters/Library/Caches/pyoxidizer/python_distributions/python.823a456d3d6e/python/install/lib/python3.10/site-packages/pkg_resources/__init__.py", line 789, in resolve
          dist = best[req.key] = env.best_match(
        File "/Users/georgewaters/Library/Caches/pyoxidizer/python_distributions/python.823a456d3d6e/python/install/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1075, in best_match
          return self.obtain(req, installer)
        File "/Users/georgewaters/Library/Caches/pyoxidizer/python_distributions/python.823a456d3d6e/python/install/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1087, in obtain
          return installer(requirement)
        File "/Users/georgewaters/Library/Caches/pyoxidizer/python_distributions/python.823a456d3d6e/python/install/lib/python3.10/site-packages/setuptools/dist.py", line 956, in fetch_build_egg
          return fetch_build_egg(self, req)
        File "/Users/georgewaters/Library/Caches/pyoxidizer/python_distributions/python.823a456d3d6e/python/install/lib/python3.10/site-packages/setuptools/installer.py", line 87, in fetch_build_egg
          wheel.install_as_egg(dist_location)
        File "/Users/georgewaters/Library/Caches/pyoxidizer/python_distributions/python.823a456d3d6e/python/install/lib/python3.10/site-packages/setuptools/wheel.py", line 96, in install_as_egg
          self._install_as_egg(destination_eggdir, zf)
        File "/Users/georgewaters/Library/Caches/pyoxidizer/python_distributions/python.823a456d3d6e/python/install/lib/python3.10/site-packages/setuptools/wheel.py", line 104, in _install_as_egg
          self._convert_metadata(zf, destination_eggdir, dist_info, egg_info)
        File "/Users/georgewaters/Library/Caches/pyoxidizer/python_distributions/python.823a456d3d6e/python/install/lib/python3.10/site-packages/setuptools/wheel.py", line 148, in _convert_metadata
          os.rename(dist_info, egg_info)
      OSError: [Errno 66] Directory not empty: '/private/var/folders/06/4dkcltv94pz6jp2fl73c646w0000gn/T/pip-install-5ccjc5qv/cairocffi_a98e862e506d4c96a615da379f0de407/.eggs/cffi-1.15.1-py3.10-macosx-10.9-x86_64.egg/cffi-1.15.1.dist-info' -> '/private/var/folders/06/4dkcltv94pz6jp2fl73c646w0000gn/T/pip-install-5ccjc5qv/cairocffi_a98e862e506d4c96a615da379f0de407/.eggs/cffi-1.15.1-py3.10-macosx-10.9-x86_64.egg/EGG-INFO'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> cairocffi

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
error[PYOXIDIZER_PYTHON_EXECUTABLE]: calling pip install

    Caused by:
        error running pip
      --> ./pyoxidizer.bzl:62:30
       |
    62 |     exe.add_python_resources(exe.pip_install(["cairocffi"]))
       |                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PythonExecutable.pip_install()

error: calling pip install

Caused by:
    error running pip
Traceback (most recent call last):
  File "/Users/georgewaters/Documents/Developer/Python/Monopoly-Probabilities/venv/bin/scriptopoly", line 8, in <module>
    sys.exit(scriptopoly.main())
  File "/Users/georgewaters/Documents/Developer/Python/Monopoly-Probabilities/scripts/scripts/scriptopoly.py", line 216, in main
    args.func(args, env)
  File "/Users/georgewaters/Documents/Developer/Python/Monopoly-Probabilities/scripts/scripts/scriptopoly.py", line 150, in pyoxidizer
    env.run(*split(PYOXIDIZER_BUILD_COMMAND))
  File "/Users/georgewaters/Documents/Developer/Python/Monopoly-Probabilities/scripts/scripts/utils.py", line 18, in run
    completed_process.check_returncode()
  File "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 457, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '('/Users/georgewaters/Documents/Developer/Python/Monopoly-Probabilities/venv/bin/pyoxidizer', 'build', '--release', '--var', 'PYOXIDIZER_BUILD_DIR', 'pyoxidizer-build')' returned non-zero exit status 1.

One other thing to note is when I install cairosvg locally with pip (version 22.2.2) it works fine.

Thanks.

[dunkmann00]

So I discovered that this issue seems to revolve around the fact that cairocffi doesn't have a wheel and when pip tries to build it for PyOxidizer it has a problem and errors out. I don't really understand why this only happens when building for PyOxidizer and not when I install it locally with pip regularly, but that is what is happening.

I did manage to find a simple workaround to this. If I change my pip_install call to use --use-pep517 when installing the problem dependency, everything works. So in this case:

exe.add_python_resources(exe.pip_install(["--use-pep517", "cariosvg"]))

works without error.

The --use-pep517 flag tells pip to use the information in pyproject.toml to first build a wheel for the library and then install from the wheel. I think at some point this will be the default behavior of pip, so you won't even have to pass this flag in, but for now, adding it seems to be a nice solution.

I'm not sure if there is any interest in trying to figure out why this error happened in the first place, but I'm going to close this issue since the workaround is trivial.