search

astral-sh / rye

a Hassle-Free Python Experience
https://rye.astral.sh
MIT License
13.6k stars 466 forks source link

UnicodeDecodeError occurs when running Python in a Rye environment with unicode strings on Windows #1196

Open sk-ys opened 2 months ago

sk-ys commented 2 months ago

Steps to Reproduce

Environment

Steps

  1. Create Rye environment with a name containing unicode strings.
  2. Change the directory to the directory created in the 1st step.
  3. Run rye sync
  4. Run rye run python

Example

rye init rye_テスト
cd rye_テスト
rye sync
rye run python

Expected Result

PS C:\temp\rye_テスト> rye run python
Python 3.12.3 (main, Apr 15 2024, 17:48:16) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

Actual Result

PS C:\temp\rye_テスト> rye run python
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1176, in exec_module
  File "<frozen site>", line 624, in <module>
  File "<frozen site>", line 607, in main
  File "<frozen site>", line 539, in venv
  File "<frozen site>", line 394, in addsitepackages
  File "<frozen site>", line 236, in addsitedir
  File "<frozen site>", line 188, in addpackage
UnicodeDecodeError: 'cp932' codec can't decode byte 0x86 in position 26: illegal multibyte sequence

Version Info

rye 0.35.0 commit: 0.35.0 (a1dbc56d4 2024-06-24) platform: windows (x86_64) self-python: cpython@3.12.3 symlink support: true uv enabled: true

Stacktrace

No response

sk-ys commented 2 months ago

While investigating this issue, I found https://github.com/python/cpython/issues/117793. So, I checked the pth file in the .venv/Lib/site-packages directory and found 2 pth files (_rye_テスト.pth, _virtualenv.pth). Each file's text encoding was UTF-8. I then changed the encoding of these files to Shift-JIS(CP932), which resolved the issue. I hope this information is helpful to you.