msys2 / MINGW-packages

Package scripts for MinGW-w64 targets to build under MSYS2.
https://packages.msys2.org
BSD 3-Clause "New" or "Revised" License
2.27k stars 1.22k forks source link

python-setuptools: DistutilsPlatformError #5155

Closed danyeaw closed 5 years ago

danyeaw commented 5 years ago

When trying to install a package locally, I am getting a UnicodeDecodeError. It looks like Python is trying to use cp1252 while loading the metadata for PEP517. I would expect that it needs to use UTF-8 in order correctly decode the metadata.

locale.getpreferredencoding() is cp1252.

$ pip3 install --user .
Processing c:/tools/msys64/home/dyeaw/gaphor
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
    Preparing wheel metadata ... error
    Complete output from command C:/tools/msys64/mingw64/bin/python3.exe C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py prepare_metadata_for_build_wheel C:/Users/dyeaw/AppData/Local/Temp/tmp237x9yfr:
    Traceback (most recent call last):
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 64, in prepare_metadata_for_build_wheel
        hook = backend.prepare_metadata_for_build_wheel
    AttributeError: module 'poetry.masonry.api' has no attribute 'prepare_metadata_for_build_wheel'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 207, in <module>
        main()
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 197, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 67, in prepare_metadata_for_build_wheel
        config_settings)
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 95, in _get_wheel_metadata_from_wheel
        whl_basename = backend.build_wheel(metadata_directory, config_settings)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\api.py", line 40, in build_wheel
        poetry, SystemEnv(Path(sys.prefix)), NullIO(), Path(wheel_directory)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\builders\wheel.py", line 48, in make_in
        wb = WheelBuilder(poetry, env, io, target_dir=directory, original=original)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\builders\wheel.py", line 38, in __init__
        super(WheelBuilder, self).__init__(poetry, env, io)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\builders\builder.py", line 42, in __init__
        self._meta = Metadata.from_package(self._package)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\metadata.py", line 50, in from_package
        meta.description = f.read()
      File "C:/tools/msys64/mingw64/lib/python3.7\encodings\cp1252.py", line 23, in decode
        return codecs.charmap_decode(input,self.errors,decoding_table)[0]
    UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 7221: character maps to <undefined>

    ----------------------------------------
Command "C:/tools/msys64/mingw64/bin/python3.exe C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py prepare_metadata_for_build_wheel C:/Users/dyeaw/AppData/Local/Temp/tmp237x9yfr" failed with error code 1 in C:/Users/dyeaw/AppData/Local/Temp/pip-req-build-clnw4_9m

Poetry is not using an encoding when it calls open(), so I forked the project and edited the metadata module. I then tried to install this version:

$ pip3 install --user .
Processing c:/tools/msys64/home/dyeaw/poetry
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  Complete output from command C:/tools/msys64/mingw64/bin/python3.exe C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py get_requires_for_build_wheel C:/Users/dyeaw/AppData/Local/Temp/tmpab8q56px:
  Traceback (most recent call last):
    File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 207, in <module>
      main()
    File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 197, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 48, in get_requires_for_build_wheel
      backend = _build_backend()
    File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 34, in _build_backend
      obj = import_module(mod_path)
    File "C:/tools/msys64/mingw64/lib/python3.7\importlib\__init__.py", line 127, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
    File "<frozen importlib._bootstrap>", line 983, in _find_and_load
    File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
    File "<frozen importlib._bootstrap>", line 983, in _find_and_load
    File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 728, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-93bwkryn/overlay/lib/python3.7/site-packages\setuptools\__init__.py", line 228, in <module>
      monkey.patch_all()
    File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-93bwkryn/overlay/lib/python3.7/site-packages\setuptools\monkey.py", line 101, in patch_all
      patch_for_msvc_specialized_compiler()
    File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-93bwkryn/overlay/lib/python3.7/site-packages\setuptools\monkey.py", line 164, in patch_for_msvc_specialized_compiler
      patch_func(*msvc9('find_vcvarsall'))
    File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-93bwkryn/overlay/lib/python3.7/site-packages\setuptools\monkey.py", line 151, in patch_params
      mod = import_module(mod_name)
    File "C:/tools/msys64/mingw64/lib/python3.7\importlib\__init__.py", line 127, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
    File "<frozen importlib._bootstrap>", line 983, in _find_and_load
    File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 728, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "C:/tools/msys64/mingw64/lib/python3.7\distutils\msvc9compiler.py", line 296, in <module>
      raise DistutilsPlatformError("VC %0.1f is not supported by this module" % VERSION)
  distutils.errors.DistutilsPlatformError: VC 6.0 is not supported by this module

I have setuptools installed via pacman only. I have also removed all of Python 3 and dependencies, along with the full /lib/python3.7 directory to make sure the distutils installed versions were blow away as well. I only used pip3 install --user to prevent any pip vs. pacman conflicts.

lazka commented 5 years ago

Poetry is not using an encoding when it calls open(), so I forked the project and edited the metadata module. I then tried to install this version:

Yeah, it's not passing an encoding here: https://github.com/sdispater/poetry/blob/b4f9e3fc9303bf4039f52ee19a1bf680684000ca/poetry/masonry/metadata.py#L49 Please file a poetry bug.

I have setuptools installed via pacman only.

Try passing --no-build-isolation to pip install

danyeaw commented 5 years ago

@lazka Thanks for the feedback. I opened an issue with Poetry, and I'll try to get a PR submitted to get it fixed.

--no-build-isolation only works for non-PEP517 installed packages, and in this case poetry doesn't have a setup.py. I get a FileNotFoundError: [Errno 2] No such file or directory: 'setup.py' error.

lazka commented 5 years ago

--no-build-isolation only works for non-PEP517 installed packages, and in this case poetry doesn't have a setup.py. I get a FileNotFoundError: [Errno 2] No such file or directory: 'setup.py' error.

Oh, I see. No idea :/

danyeaw commented 5 years ago

@lazka What are your thoughts on an overall strategy on fixing these types of Python issues along with #5001. It seems like Python is changing faster than we can patch downstream. Would it at least be worthwhile to start a discussion on the python-dev mailing list about where there is interest in the Python community providing more official support for MSYS2?

lazka commented 5 years ago

What would be the goal of starting a discussion there? No one there is using it and probably doesn't plan to.

We could spin the Python3 package out into a CPython fork with patches on top (+ CI maybe?) and try to push some simple (not MSYS2 specific) things upstream. That would probably make it more visible as its own "platform". But, time.... :)

For the setuptools error, we could try to patch msvc9compiler.py (which fails here because it tries to detect the MSVC version Python was built with) to not fail to import, maybe we can then drop the setuptools patch.

lazka commented 5 years ago

For the setuptools error, we could try to patch msvc9compiler.py (which fails here because it tries to detect the MSVC version Python was built with) to not fail to import, maybe we can then drop the setuptools patch.

This is what I did now in #5260

danyeaw commented 5 years ago

@lazka Thanks so much for resolving this ❤️

lazka commented 5 years ago

Note that installation will still be broken in some cases, not sure when this is relevant for example: https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-python-setuptools/0001-mingw-python-fix.patch