search

iterative / dvc

🦉 ML Experiments and Data Management with Git
https://dvc.org
Apache License 2.0
13.41k stars 1.16k forks source link

Unknown encoding error (cp65001) in windows exe version #2165

Closed LexSong closed 5 years ago

LexSong commented 5 years ago

DVC version: dvc-0.41.3.exe windows exe

Error message:

C:\Users\LexSong> dvc

C:\PROGRA~2\DATAVE~1\aliyunsdkcore\vendored\requests\packages\urllib3\contrib\socks.py:37: DependencyWarning: SOCKS support in urllib3 requires the installation of optional dependencies: specifically, PySocks.  For more information, see https://urllib3.readthedocs.io/en/latest/contrib.html#socks-proxies
Traceback (most recent call last):
  File "logging\__init__.py", line 868, in emit
  File "logging\__init__.py", line 741, in format
  File "dvc\logger.py", line 51, in format
  File "dvc\logger.py", line 149, in _progress_aware
  File "dvc\progress.py", line 36, in clearln
  File "dvc\progress.py", line 117, in _print
  File "site-packages\colorama\ansitowin32.py", line 41, in write
  File "site-packages\colorama\ansitowin32.py", line 162, in write
  File "site-packages\colorama\ansitowin32.py", line 187, in write_and_convert
  File "site-packages\colorama\ansitowin32.py", line 195, in write_plain_text
LookupError: unknown encoding: cp65001
Logged from file cli.py, line 83
Traceback (most recent call last):
  File "logging\__init__.py", line 868, in emit
  File "logging\__init__.py", line 741, in format
  File "dvc\logger.py", line 51, in format
  File "dvc\logger.py", line 149, in _progress_aware
  File "dvc\progress.py", line 36, in clearln
  File "dvc\progress.py", line 117, in _print
  File "site-packages\colorama\ansitowin32.py", line 41, in write
  File "site-packages\colorama\ansitowin32.py", line 162, in write
  File "site-packages\colorama\ansitowin32.py", line 187, in write_and_convert
  File "site-packages\colorama\ansitowin32.py", line 195, in write_plain_text
LookupError: unknown encoding: cp65001
Logged from file main.py, line 48

There is an encoding error in colorama when the code page is cp65001(UTF-8). There is no issue when I install with pip install on Anaconda.

shcheklein commented 5 years ago

@LexSong could you try to do set PYTHONIOENCODING=UTF-8 in your terminal before running dvc?

efiop commented 5 years ago

Hi @LexSong ! You are running a default windows terminal, right? And when running on Anaconda, you are using their own terminal? @shcheklein is right, setting that env var should help. Also, if it does, then it is most likely windows-terminal-specific error. To solve it we would need to include https://github.com/Drekin/win-unicode-console into our dependencies when running/building on windows.

Thanks for the feedback! :)

LexSong commented 5 years ago

And when running on Anaconda, you are using their own terminal?

No, I'm using the default windows 10 terminal, too.

set PYTHONIOENCODING=UTF-8

I'll see if this works.

LexSong commented 5 years ago

set PYTHONIOENCODING=UTF-8 doesn't work.

efiop commented 5 years ago

@LexSong I've added a package to our dependencies in https://github.com/iterative/dvc/commit/c7ca83e226f5b10c2b986307820a74ba846027cc , which might help with that. Would you be ok with using anaconda for now? If so, we will soon build a new windows package with that package included and will notify you to try it out, so you could confirm or deny that it works for you.

LexSong commented 5 years ago

By the way, my anaconda environment is here:

name: dvc
channels:
  - defaults
  - conda-forge
dependencies:
  - asn1crypto=0.24.0=py37_0
  - ca-certificates=2019.5.15=0
  - certifi=2019.6.16=py37_0
  - cffi=1.12.3=py37h7a1dbc1_0
  - cryptography=2.7=py37h7a1dbc1_0
  - dulwich=0.19.11=py37hfa6e2cd_0
  - idna=2.8=py37_0
  - openssl=1.1.1c=he774522_1
  - pip=19.1.1=py37_0
  - pycparser=2.19=py37_0
  - pyopenssl=19.0.0=py37_0
  - pysocks=1.7.0=py37_0
  - python=3.7.3=h8c8aaf0_1
  - setuptools=41.0.1=py37_0
  - six=1.12.0=py37_0
  - sqlite=3.28.0=he774522_0
  - urllib3=1.25.3=py37_0
  - vc=14.1=h0510ff6_4
  - vs2015_runtime=14.15.26706=h3a45250_4
  - wheel=0.33.4=py37_0
  - win_inet_pton=1.1.0=py37_0
  - wincertstore=0.2=py37_0
  - pip:
    - appdirs==1.4.3
    - asciimatics==1.11.0
    - chardet==3.0.4
    - colorama==0.4.1
    - configobj==5.0.6
    - configparser==3.7.4
    - contextlib2==0.5.5
    - decorator==4.4.0
    - distro==1.4.0
    - dvc==0.41.3
    - future==0.17.1
    - gitdb2==2.0.5
    - gitpython==2.1.11
    - grandalf==0.6
    - humanize==0.5.1
    - inflect==2.1.0
    - jsonpath-ng==1.4.3
    - nanotime==0.5.2
    - networkx==2.3
    - pillow==6.0.0
    - ply==3.11
    - pyasn1==0.4.5
    - pyfiglet==0.8.post1
    - pyparsing==2.4.0
    - pypiwin32==223
    - pywin32==224
    - requests==2.22.0
    - ruamel-yaml==0.15.97
    - schema==0.7.0
    - smmap2==2.0.5
    - treelib==1.5.5
    - wcwidth==0.1.7
    - zc-lockfile==1.4
prefix: C:\Miniconda3\envs\dvc

Maybe it will be helpful.

LexSong commented 5 years ago

OK, I think the issue may be caused by Python 2.7. Good luck to fix it. I'll keep using Python 3 for now.

efiop commented 5 years ago

@LexSong Oh, good point! We are still using python2 to build binaries with pyinstaller. I'll adjust it to use python3 shortly too. Thanks! :)

LexSong commented 5 years ago

Related StackOverflow question

efiop commented 5 years ago

Ok, now we do build using py3 with that package included, which should be released soon. I'll close this for now. Please feel free to reopen if it still persists.

LexSong commented 5 years ago

Thank you! I can confirm this issue have been fixed in 0.50.1 released exe.