search

damnever / pigar

:coffee: A tool to generate requirements.txt for Python project, and more than that. (IT IS NOT A PACKAGE MANAGEMENT TOOL)
https://damnever.github.io/pigar/
BSD 3-Clause "New" or "Revised" License
1.64k stars 90 forks source link

Pigar 2.0.0 not working when running from poetry #125

Closed h3llrais3r closed 1 year ago

h3llrais3r commented 1 year ago

Installed latest 2.0.0 version via poetry. Running it via poetry in venv as I don't want to install it globally. image

EDIT: Running directly in venv gives the same error. image

Previous version was working perfectly.

damnever commented 1 year ago

Need more information about your virtual environment to reproduce it, would you mind posting the output of pip freeze?

You can also run pigar -l DEBUG gen to see which distribution causes the problem.

And you can try this branch as well: pip install git+https://github.com/damnever/pigar.git@fix/125 -U

yochananmarqos commented 1 year ago

I receive a similar error with 2.0.0 & 2.0.1 running pigar generate:

Traceback (most recent call last):
  File "/usr/bin/pigar", line 33, in <module>
    sys.exit(load_entry_point('pigar==2.0.1', 'console_scripts', 'pigar')())
  File "/usr/lib/python3.10/site-packages/pigar/__main__.py", line 474, in main
    cli()
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/pigar/__main__.py", line 235, in generate
    analyzer = RequirementsAnalyzer(project_path)
  File "/usr/lib/python3.10/site-packages/pigar/core.py", line 36, in __init__
    self._installed_dists = installed_distributions()
  File "/usr/lib/python3.10/site-packages/pigar/dist.py", line 201, in installed_distributions
    req = FrozenRequirement.from_dist(distribution)
  File "/usr/lib/python3.10/site-packages/pigar/dist.py", line 134, in from_dist
    os.path.commonpath([code_file_dir, file]) ==
  File "/usr/lib/python3.10/posixpath.py", line 526, in commonpath
    raise ValueError("Can't mix absolute and relative paths") from None
ValueError: Can't mix absolute and relative paths

This is using the Arch PKGBUILD on Manjaro (unstable branch).

The tests also fail:

python -m unittest discover pigar/tests/ -t . -v
test_check (pigar.tests.test_cli.CliTests) ... FAIL
test_generate (pigar.tests.test_cli.CliTests) ... FAIL
test_search (pigar.tests.test_cli.CliTests) ... FAIL
test_add (pigar.tests.test_core.LocatableRequirementsTest) ... ok
test_add (pigar.tests.test_core.LocationsTests) ... ok
test_extend (pigar.tests.test_core.LocationsTests) ... ok
test_sorted_items (pigar.tests.test_core.LocationsTests) ... ok
test_analyze_requirements (pigar.tests.test_core.RequirementsAnalyzerTests) ... ERROR
test_stdlib_deprecated_since_py3_12 (pigar.tests.test_core.StdlibTest) ... ok
test_stdlibs (pigar.tests.test_core.StdlibTest) ... ok
test_stdlibs_added_since_py3_11 (pigar.tests.test_core.StdlibTest) ... skipped '< Py3.11'
test_stdlibs_added_since_py3_8 (pigar.tests.test_core.StdlibTest) ... ok
test_stdlibs_added_since_py3_9 (pigar.tests.test_core.StdlibTest) ... ok
test_db (pigar.tests.test_db.DBTests) ... ok
test_parse_urls_from_html (pigar.tests.test_dist.HTMLParserTest) ... ok
test_compare_version (pigar.tests.test_helpers.CompareVersionTests) ... ok
test_sort_versions (pigar.tests.test_helpers.CompareVersionTests) ... ok
test_attr_error (pigar.tests.test_helpers.DictTests) ... ok
test_getattr (pigar.tests.test_helpers.DictTests) ... ok
test_init_with_dict (pigar.tests.test_helpers.DictTests) ... ok
test_init_with_kwargs (pigar.tests.test_helpers.DictTests) ... ok
test_init_with_zip_list (pigar.tests.test_helpers.DictTests) ... ok
test_parse_requirements (pigar.tests.test_helpers.ParseReqsTest) ... ok
test_custom_headers (pigar.tests.test_helpers.PrintTableTests) ... ok
test_default_headers (pigar.tests.test_helpers.PrintTableTests) ... ok
test_tar (pigar.tests.test_unpack.TopLevelTests) ... ok
test_zip (pigar.tests.test_unpack.TopLevelTests) ... ok

======================================================================
ERROR: test_analyze_requirements (pigar.tests.test_core.RequirementsAnalyzerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/pigar/src/pigar-2.0.1/pigar/tests/test_core.py", line 69, in test_analyze_requirements
    analyzer = RequirementsAnalyzer(self._path)
  File "/build/pigar/src/pigar-2.0.1/pigar/core.py", line 36, in __init__
    self._installed_dists = installed_distributions()
  File "/build/pigar/src/pigar-2.0.1/pigar/dist.py", line 201, in installed_distributions
    req = FrozenRequirement.from_dist(distribution)
  File "/build/pigar/src/pigar-2.0.1/pigar/dist.py", line 134, in from_dist
    os.path.commonpath([code_file_dir, file]) ==
  File "/usr/lib/python3.10/posixpath.py", line 526, in commonpath
    raise ValueError("Can't mix absolute and relative paths") from None
ValueError: Can't mix absolute and relative paths

======================================================================
FAIL: test_check (pigar.tests.test_cli.CliTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/pigar/src/pigar-2.0.1/pigar/tests/test_cli.py", line 63, in test_check
    self.assertEqual(result.exit_code, 0, result.output)
AssertionError: 1 != 0 : 

======================================================================
FAIL: test_generate (pigar.tests.test_cli.CliTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/pigar/src/pigar-2.0.1/pigar/tests/test_cli.py", line 42, in test_generate
    self.assertEqual(result.exit_code, 0, result.output)
AssertionError: 1 != 0 : 

======================================================================
FAIL: test_search (pigar.tests.test_cli.CliTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/pigar/src/pigar-2.0.1/pigar/tests/test_cli.py", line 68, in test_search
    self.assertEqual(result.exit_code, 0, result.output)
AssertionError: 1 != 0 : 

----------------------------------------------------------------------
Ran 27 tests in 0.196s

FAILED (failures=3, errors=1, skipped=1)
damnever commented 1 year ago

@yochananmarqos Could you post the output of the following command?

python -c "from distlib.database import DistributionPath; [print(dist.name, list(dist.list_installed_files())) for dist in DistributionPath(include_egg=True).get_distributions()]"
h3llrais3r commented 1 year ago

pip freeze output:

aiohttp==3.8.3
aiosignal==1.3.1
appdirs==1.4.4
async-timeout==4.0.2
asynctest==0.13.0
attrs==22.1.0
auditok==0.1.5
autocommand==2.2.1
autopep8==2.0.0
-e git+https://github.com/h3llrais3r/Auto-Subliminal.git@f00c511c035387303b0c99585a0a89bbbfa3f2af#egg=autosubliminal
babelfish==0.6.0
beautifulsoup4==4.11.1
boto==2.49.0
CacheControl==0.12.11
cchardet==2.1.7
certifi==2022.9.24
chardet==5.0.0
charset-normalizer==2.1.1
cheroot==8.6.0
CherryPy==18.8.0
click==8.1.3
colorama==0.4.5
commonmark==0.9.1
contextlib2==21.6.0
coverage==6.5.0
coveralls==3.3.1
deathbycaptcha==0.1.1
decorator==5.1.1
diskcache==5.4.0
distlib==0.3.6
distro==1.7.0
docopt==0.6.2
dogpile.cache==1.1.8
enzyme==0.4.1
exceptiongroup==1.0.0rc9
fastjsonschema==2.16.1
ffmpeg-python==0.2.0
ffsubsync==0.4.20
flake8==5.0.4
frozenlist==1.3.3
future==0.18.2
gitdb==4.0.9
GitPython==3.1.29
gntp==1.0.3
guessit==3.5.0
html5lib==1.1
idna==3.4
imdbpie @ git+https://github.com/h3llrais3r/imdb-pie.git@75c300af1bf87c49baef09f17002a3e35b0ed3cc
importlib-metadata==4.2.0
importlib-resources==5.9.0
inflect==6.0.0
iniconfig==1.1.1
isort==5.10.1
jaraco.classes==3.2.2
jaraco.collections==3.5.2
jaraco.context==4.1.2
jaraco.functools==3.5.1
jaraco.text==3.9.1
jsonpickle==3.0.0
jsonschema==4.16.0
jupyter-core==4.11.1
langdetect==1.0.9
mccabe==0.7.0
more-itertools==8.14.0
msgpack==1.0.4
multidict==6.0.2
nbformat==5.5.0
numpy==1.21.1
oauthlib==3.2.1
packaging==21.3
pbr==5.10.0
pep517==0.13.0
pigar==2.0.0
pkgutil_resolve_name==1.3.10
platformdirs==2.5.2
pluggy==1.0.0
portend==3.1.0
py==1.11.0
pycodestyle==2.9.1
pydantic==1.10.2
pyflakes==2.5.0
Pygments==2.13.0
pyparsing==3.0.9
pyrsistent==0.18.1
pysrt==1.1.2
pysubs2==1.4.3
pytest==7.2.0
pytest-mock==3.10.0
python-anticaptcha==1.0.0
python-dateutil==2.8.2
python-twitter==3.5
pytz==2022.2.1
pywin32==304
PyYAML==6.0
rarfile==4.0
rebulk==3.1.0
requests==2.28.1
requests-mock==1.10.0
requests-oauthlib==1.3.1
resolvelib==0.8.1
rich==12.5.1
schema==0.7.5
simplejson==3.18.0
six==1.16.0
smmap==5.0.0
soupsieve==2.3.2.post1
srt==3.5.2
stevedore==3.5.0
subliminal @ git+https://github.com/Diaoul/subliminal.git@6ac2fa23ee5baa7d8452552edaa7c4a8a00d237a
tailer==0.4.1
tempora==5.0.2
tenacity==8.1.0
toml==0.10.2
tomli==2.0.1
tqdm==4.64.1
traitlets==5.4.0
trans==2.1.0
tvdb-api @ git+https://github.com/h3llrais3r/tvdb_api.git@1210df9dd5869ccc5b63149b1b80630310a14f40
typing_extensions==4.3.0
Unidecode==1.3.6
urllib3==1.26.12
vcrpy==4.2.1
webencodings==0.5.1
webrtcvad-wheels==2.0.10.post2
wrapt==1.14.1
ws4py==0.5.1
yarl==1.8.1
zc.lockfile==2.0
zipp==3.8.1
h3llrais3r commented 1 year ago

Same issue with:

yochananmarqos commented 1 year ago

@damnever Here's the output.

EDIT: The tests still fail applying bf53bcd9 to 2.0.1.

damnever commented 1 year ago

@h3llrais3r I can't reproduce the issue with those requirements, maybe something related to the Windows platform..

@yochananmarqos So weird, I didn't find anything special either.

You may try this branch again: pip install git+https://github.com/damnever/pigar.git@fix/125 -U

h3llrais3r commented 1 year ago

@damnever it has to do with the installation via poetry... When I'm installing in a clean venv with the pip freeze output, it works. When it's installed via poetry (on Windows) in a clean venv, it has issues. Since I'm using poetry for my build environment, I need to stick with pigar version < 2 until this is fixed.

Can you try with a poetry install on Windows?

damnever commented 1 year ago

@h3llrais3r I don't have a Windows environment, so I have to guess the causes for now..

I have published another release, hopefully, pip install pigar==2.0.2 could fix it.

h3llrais3r commented 1 year ago

Will test today and provide feedback.

h3llrais3r commented 1 year ago

@damnever, I confirm that 2.0.2 has fixed the issue! image

Thanks!

yochananmarqos commented 1 year ago

2.0.2 seems to have fixed the issue. However, one test still fails:

test_check (pigar.tests.test_cli.CliTests) ... ok
test_generate (pigar.tests.test_cli.CliTests) ... FAIL
test_search (pigar.tests.test_cli.CliTests) ... ok
test_add (pigar.tests.test_core.LocatableRequirementsTest) ... ok
test_add (pigar.tests.test_core.LocationsTests) ... ok
test_extend (pigar.tests.test_core.LocationsTests) ... ok
test_sorted_items (pigar.tests.test_core.LocationsTests) ... ok
test_analyze_requirements (pigar.tests.test_core.RequirementsAnalyzerTests) ... ok
test_stdlib_deprecated_since_py3_12 (pigar.tests.test_core.StdlibTest) ... ok
test_stdlibs (pigar.tests.test_core.StdlibTest) ... ok
test_stdlibs_added_since_py3_11 (pigar.tests.test_core.StdlibTest) ... skipped '< Py3.11'
test_stdlibs_added_since_py3_8 (pigar.tests.test_core.StdlibTest) ... ok
test_stdlibs_added_since_py3_9 (pigar.tests.test_core.StdlibTest) ... ok
test_db (pigar.tests.test_db.DBTests) ... ok
test_parse_urls_from_html (pigar.tests.test_dist.HTMLParserTest) ... ok
test_compare_version (pigar.tests.test_helpers.CompareVersionTests) ... ok
test_sort_versions (pigar.tests.test_helpers.CompareVersionTests) ... ok
test_attr_error (pigar.tests.test_helpers.DictTests) ... ok
test_getattr (pigar.tests.test_helpers.DictTests) ... ok
test_init_with_dict (pigar.tests.test_helpers.DictTests) ... ok
test_init_with_kwargs (pigar.tests.test_helpers.DictTests) ... ok
test_init_with_zip_list (pigar.tests.test_helpers.DictTests) ... ok
test_parse_requirements (pigar.tests.test_helpers.ParseReqsTest) ... ok
test_custom_headers (pigar.tests.test_helpers.PrintTableTests) ... ok
test_default_headers (pigar.tests.test_helpers.PrintTableTests) ... ok
test_tar (pigar.tests.test_unpack.TopLevelTests) ... ok
test_zip (pigar.tests.test_unpack.TopLevelTests) ... ok

======================================================================
FAIL: test_generate (pigar.tests.test_cli.CliTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/storage/pkgbuilds/pigar/src/pigar-2.0.2/pigar/tests/test_cli.py", line 48, in test_generate
    self.assertEqual(len(expected), len(actual))
AssertionError: 118 != 128

----------------------------------------------------------------------
Ran 27 tests in 34.597s

FAILED (failures=1, skipped=1)
damnever commented 1 year ago

@h3llrais3r @yochananmarqos thanks for your feedback.

@yochananmarqos maybe because some optional(try .. except ImportError: ..) distributions have been installed in your environment, such as keyring. pigar will include them if they are installed otherwise they will be ignored silently.