search

mesonbuild / meson

The Meson Build System
http://mesonbuild.com
Apache License 2.0
5.3k stars 1.52k forks source link

UnicodeDecodeError when installing scipy #13154

Closed Max-Ole closed 2 weeks ago

Max-Ole commented 2 weeks ago

Hi all,

I'm on Windows 11 and I have set up a fresh Python 3.11 virtual environment and used pip to install ultralytics. This in turn installs scipy, which results in the UnicodeDecodeError error:

File "<frozen codecs>", line 322, in decode
      UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc4 in position 54: invalid continuation byte

This seems to happen together with a failure of metadata generation with a file called pyproject.toml. Note that my working directory path contains non-ascii characters, if that is of any use.

To Reproduce:

I created a new venv: pyenv local 3.11.0b4 python3 -m venv myPyEnv I activated the venv with a .bat script containing: cmd /k "cd /d myPyEnv\Scripts && activate.bat && cd ..\.." Then I ran: pip install ultralytics

Terminal output:

C:\Users\maxol\MEINS\STUDIUM\FÄCHER\Master Arbeit\Master_Thesis_Literature\Sandbox>cmd /k "cd /d myPyEnv\Scripts && activate.bat && cd ..\.."

(myPyEnv) C:\Users\maxol\MEINS\STUDIUM\FÄCHER\Master Arbeit\Master_Thesis_Literature\Sandbox>pip list
Package    Version
---------- -------
pip        24.0
setuptools 58.1.0

(myPyEnv) C:\Users\maxol\MEINS\STUDIUM\FÄCHER\Master Arbeit\Master_Thesis_Literature\Sandbox>pip install ultralytics
Collecting ultralytics
  Downloading ultralytics-8.2.4-py3-none-any.whl.metadata (40 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.6/40.6 kB 2.0 MB/s eta 0:00:00
Collecting matplotlib>=3.3.0 (from ultralytics)
  Downloading matplotlib-3.8.4.tar.gz (35.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 35.9/35.9 MB 8.2 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting opencv-python>=4.6.0 (from ultralytics)
  Downloading opencv_python-4.9.0.80-cp37-abi3-win32.whl.metadata (20 kB)
Collecting pillow>=7.1.2 (from ultralytics)
  Downloading pillow-10.3.0-cp311-cp311-win32.whl.metadata (9.4 kB)
Collecting pyyaml>=5.3.1 (from ultralytics)
  Downloading PyYAML-6.0.1-cp311-cp311-win32.whl.metadata (2.1 kB)
Collecting requests>=2.23.0 (from ultralytics)
  Using cached requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting scipy>=1.4.1 (from ultralytics)
  Downloading scipy-1.13.0.tar.gz (57.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.2/57.2 MB 6.9 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 2
  ╰─> [25 lines of output]
      + meson setup C:\Users\maxol\AppData\Local\Temp\pip-install-7cn8o__9\scipy_e3b2e9eac7c64e81b00f14057e4aaa67 C:\Users\maxol\AppData\Local\Temp\pip-install-7cn8o__9\scipy_e3b2e9eac7c64e81b00f14057e4aaa67\.mesonpy-jv6pmawh -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=C:\Users\maxol\AppData\Local\Temp\pip-install-7cn8o__9\scipy_e3b2e9eac7c64e81b00f14057e4aaa67\.mesonpy-jv6pmawh\meson-python-native-file.ini
      Traceback (most recent call last):
        File "C:\Users\maxol\AppData\Local\Temp\pip-build-env-hs_2kreh\overlay\Lib\site-packages\mesonbuild\mesonmain.py", line 186, in run
          return options.run_func(options)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\maxol\AppData\Local\Temp\pip-build-env-hs_2kreh\overlay\Lib\site-packages\mesonbuild\msetup.py", line 363, in run
          app.generate()
        File "C:\Users\maxol\AppData\Local\Temp\pip-build-env-hs_2kreh\overlay\Lib\site-packages\mesonbuild\msetup.py", line 180, in generate
          env = environment.Environment(self.source_dir, self.build_dir, self.options)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\maxol\AppData\Local\Temp\pip-build-env-hs_2kreh\overlay\Lib\site-packages\mesonbuild\environment.py", line 591, in __init__
          config = coredata.parse_machine_files(self.coredata.config_files, self.source_dir)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\maxol\AppData\Local\Temp\pip-build-env-hs_2kreh\overlay\Lib\site-packages\mesonbuild\coredata.py", line 1164, in parse_machine_files
          parser = MachineFileParser(filenames, sourcedir)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "C:\Users\maxol\AppData\Local\Temp\pip-build-env-hs_2kreh\overlay\Lib\site-packages\mesonbuild\coredata.py", line 1099, in __init__
          content = f.read()
                    ^^^^^^^^
        File "<frozen codecs>", line 322, in decode
      UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc4 in position 54: invalid continuation byte

      ERROR: Unhandled python exception

          This is a Meson bug and should be reported!
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

(myPyEnv) C:\Users\maxol\MEINS\STUDIUM\FÄCHER\Master Arbeit\Master_Thesis_Literature\Sandbox>
Max-Ole commented 2 weeks ago

Update:

I was able to install scipy now by running the steps below. I had success on a different computer where I didn't use special characters and hadn't tried the installation before. With these steps it then also worked on the original computer:

  1. Created a new venv in a directory without special characters in the path.
  2. Upgraded pip.
  3. Installed scipy without using the cache (--no-cache-dir)

Note that this time there was no mention of "Preparing metadata". This might not have fixed but just skipped the meson steps.

Terminal output:

C:\Users\maxol\Desktop\MasterThese_venv>pyenv versions
  3.11.0b4-win32

C:\Users\maxol\Desktop\MasterThese_venv>pyenv local 3.11.0b4-win32

C:\Users\maxol\Desktop\MasterThese_venv>pyenv local
3.11.0b4-win32

C:\Users\maxol\Desktop\MasterThese_venv>python -m venv thesis_venv

C:\Users\maxol\Desktop\MasterThese_venv>thesis_venv\Scripts\activate

(thesis_venv) C:\Users\maxol\Desktop\MasterThese_venv>pip list
Package    Version
---------- -------
pip        23.2.1
setuptools 65.5.0

[notice] A new release of pip is available: 23.2.1 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip

(thesis_venv) C:\Users\maxol\Desktop\MasterThese_venv>python.exe -m pip install --upgrade pip
Requirement already satisfied: pip in c:\users\maxol\desktop\masterthese_venv\thesis_venv\lib\site-packages (23.2.1)
Collecting pip
  Obtaining dependency information for pip from https://files.pythonhosted.org/packages/8a/6a/19e9fe04fca059ccf770861c7d5721ab4c2aebc539889e97c7977528a53b/pip-24.0-py3-none-any.whl.metadata
  Using cached pip-24.0-py3-none-any.whl.metadata (3.6 kB)
Using cached pip-24.0-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.2.1
    Uninstalling pip-23.2.1:
      Successfully uninstalled pip-23.2.1
Successfully installed pip-24.0

(thesis_venv) C:\Users\maxol\Desktop\MasterThese_venv>pip install scipy --no-cache-dir
Collecting scipy
  Downloading scipy-1.13.0-cp311-cp311-win_amd64.whl.metadata (60 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.6/60.6 kB 645.4 kB/s eta 0:00:00
Collecting numpy<2.3,>=1.22.4 (from scipy)
  Downloading numpy-1.26.4-cp311-cp311-win_amd64.whl.metadata (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.0/61.0 kB ? eta 0:00:00
Downloading scipy-1.13.0-cp311-cp311-win_amd64.whl (46.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.2/46.2 MB 7.0 MB/s eta 0:00:00
Downloading numpy-1.26.4-cp311-cp311-win_amd64.whl (15.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.8/15.8 MB 7.0 MB/s eta 0:00:00
Installing collected packages: numpy, scipy
Successfully installed numpy-1.26.4 scipy-1.13.0

(thesis_venv) C:\Users\maxol\Desktop\MasterThese_venv>
dnicolodi commented 2 weeks ago

I find it odd that your first installation attempt resulted in pip trying to build scipy from source, but not the second.

Anyhow, there is a bug in meson-python where we write the native file with the wrong encoding. Fix waiting in https://github.com/mesonbuild/meson-python/pull/623

Max-Ole commented 2 weeks ago

I noticed that even in a fresh venv, pip cache list shows wheels that I used elsewhere on the computer. Running it now I don't see anything related to scipy in the cache, but just maybe the --no-cache-dir option made the change.

Anyway, it's working and thanks for the fix!