search

bndr / pipreqs

pipreqs - Generate pip requirements.txt file based on imports of any project. Looking for maintainers to move this project forward.
Apache License 2.0
6.38k stars 388 forks source link

Issue when using with the black (formatting) module #363

Open vishalnandagopal opened 1 year ago

vishalnandagopal commented 1 year ago 
ERROR: Failed on file: .\.venv\Lib\site-packages\black\linegen.py
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\pipreqs\pipreqs.py", line 131, in get_all_imports
    raise exc
  File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\site-packages\pipreqs\pipreqs.py", line 117, in get_all_imports
    tree = ast.parse(contents)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\V\AppData\Local\Programs\Python\Python311\Lib\ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<unknown>", line 412
    Ø: Set[str] = set()
     ^
SyntaxError: invalid character '˜' (U+02DC)

This seems to be occurring due to this line in this file - https://github.com/psf/black/blob/a552f7096a9f6e016c9bb1df1e0a77a17caeec1c/src/black/linegen.py#L471 due to the weird variable name.

Because Black is such a popular library, and the variable name seems to be valid, any workarounds for using pipreqs to generate a requirements.txt file?

vishalnandagopal commented 1 year ago

Does pipreqs not parse unicode characters? Ø seems to be a valid unicode character but not a basic ASCII character.

I tried to fix it in black itself, but it was rightly closed since the issue is not in black.

JelleZijlstra commented 1 year ago

Seems like pipreqs doesn't use the correct encoding (utf-8 by default) for opening Python files. The error message posted above has mojibake, suggesting the file was opened in something like Latin-1 instead of UTF-8.