search

bndr / pipreqs

pipreqs - Generate pip requirements.txt file based on imports of any project. Looking for maintainers to move this project forward.
Apache License 2.0
6.38k stars 388 forks source link

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb1 in position 81: invalid start byte #372

Closed kuelite closed 11 months ago

kuelite commented 1 year ago

version: 0.4.13 python: 3.8.10

(virtualEnv) PS D:\Python\pythonProject1> pipreqs ./ --encoding=utf8 --debug --force
Traceback (most recent call last): File "D:\Python3810\lib\runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "D:\Python3810\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "D:\Python\virtualEnv\Scripts\pipreqs.exe__main__.py", line 7, in File "D:\Python\virtualEnv\lib\site-packages\pipreqs\pipreqs.py", line 528, in main init(args) File "D:\Python\virtualEnv\lib\site-packages\pipreqs\pipreqs.py", line 455, in init candidates = get_all_imports(input_path, File "D:\Python\virtualEnv\lib\site-packages\pipreqs\pipreqs.py", line 115, in get_all_imports contents = f.read() File "D:\Python3810\lib\codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb1 in position 81: invalid start byte

How to solve the annoying problem？

ClaireCJS commented 1 year ago

Seconded.

I have a unicode-scrubber with a lot of differently-encoded characters in it, and I can't generate requirements.txt because of this error.

pipreqs . Traceback (most recent call last): File "C:\ProgramData\anaconda3\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\ProgramData\anaconda3\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "C:\ProgramData\anaconda3\Scripts\pipreqs.exe\__main__.py", line 7, in <module> File "C:\ProgramData\anaconda3\lib\site-packages\pipreqs\pipreqs.py", line 528, in main init(args) File "C:\ProgramData\anaconda3\lib\site-packages\pipreqs\pipreqs.py", line 455, in init candidates = get_all_imports(input_path, File "C:\ProgramData\anaconda3\lib\site-packages\pipreqs\pipreqs.py", line 115, in get_all_imports contents = f.read() File "C:\ProgramData\anaconda3\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 10266: character maps to <undefined>

alan-barzilay commented 11 months ago

You have to figure out your file encoding to fix this issue, it is using a character not recognized by the default encoding. A good first attempt is to try utf-8, if it doesnt work maybe try an encoding from your region. We cant really help you with this issue, sorry