search

pivotal / LicenseFinder

Find licenses for your project's dependencies.
MIT License
1.72k stars 339 forks source link

LF fails to recognize installed pip deps if name differs on pypi #901

Open doosuu opened 2 years ago

doosuu commented 2 years ago

Environment:

Observation: license_finder_pip.py exits with DistributionNotFound even though the dependency is installed. This is related to canonicalization of project names. If the requirements.txt is modified to replace dashes with dots, it works.

https://pypi.org/project/ruamel.yaml.clib/ reports the original project name with dots. But both formats are supported by pip install.

The big issue here is that we are generating requirements.txt files automatically using pip-tools (pip-compile). These will always report ruamel-yaml-clib with dashes instead of dots.

I'm guessing license finder should be able to support the different formats supported by pip.

cf-gitbot commented 2 years ago

We have created an issue in Pivotal Tracker to manage this. Unfortunately, the Pivotal Tracker project is private so you may be unable to view the contents of the story.

The labels on this github issue will be updated when the story is started.

xtreme-shane-lattanzio commented 2 years ago

@sdo2hi So I managed to repro this by using the LF docker image, making a requirements.txt with just ruamel-yaml-clib==0.2.6 inside with dots instead and using pip3 install -r requirements.txt followed by python3 ../bin/license_finder_pip.py requirements.txt That worked and then I swapped it back to dashes which failed.

Narrowing it down, the line failing is when it tried to parse with pkg_resources.working_set.resolve(reqs). This means the issue is actually compatibility with the pip versions as per this github issue:https://github.com/jazzband/pip-tools/issues/1576

I don't know if I'm crazy but I cannot get the 62.0 version of the setuptools installed which is what that thread suggests. If you want to try this out and potentially make a change to the docker image, it should get this green. It may need a pip3 version bump to get the new version of setuptools but I am not sure. Let me know if you can throw up a PR and we will get this merged!