search

bndr / pipreqs

pipreqs - Generate pip requirements.txt file based on imports of any project. Looking for maintainers to move this project forward.
Apache License 2.0
6.38k stars 388 forks source link

Pipreqs adds wrong package to requirements.txt #360

Open artemgur opened 1 year ago

artemgur commented 1 year ago

In my project, I use kafka-python package. When pipreqs generates requirements.txt, it adds kafka package in addition to kafka-python package. kafka package is not used in the project, it's not even installed.

Arfius commented 1 year ago

I have the same problem.

p4r4xor commented 1 year ago

I suppose pipreqs doesn't support hyphenated packages yet.

Godd67 commented 1 year ago

I have similar issue with azure-storage. It adds azure-storage==0.37, which is deprecated and cannot be installed instead of azure-storage-blob==12.15.0, that is actually installed and used. Is there a workaround for this or should I switch back to pip freeze to create requirements files?

CwjXFH commented 1 year ago

It looks like the same problem as sklearn not correctly formed

kevindugan commented 1 year ago

It adds azure-storage==0.37, which is deprecated and cannot be installed instead of azure-storage-blob==12.15.0, that is actually installed and used. Is there a workaround for this or should I switch back to pip freeze to create requirements files?

I'm having the same issue with azure related modules. It seems like there are a few assumptions in the package deduction that are getting in the way.

  1. For azure specifically, there are multiple entries in the mapping with the 'azure' key. Meaning that subsequent entries for 'azure' will overwrite previous entries. Hence, why azure-storage is showing up.
  2. The get_all_imports function assumes that we only want the top-level module. They give the example django.conf should only import django. This assumption is broken for several of the azure modules, eg azure.functions actually should point to the azure-functions project.

Currently I don't have a great proposal for how to redesign the mapping between module imports and pypi project names. I think this is a recurring issue that the maintainers have identified. My current thought is there may be a way to programatically generate the mapping file by queries to the pypi package api, though I don't have a working prototype yet.

fs34a commented 1 year ago

I had the same problem. I have a private python module locates in folder lib, then pipreqs added lib into requirement.txt while this lib acctully refers to https://pypi.org/project/lib/ and confused users that install with this txt file.

bobfirestone commented 1 year ago

I had the same problem. I have a private python module locates in folder lib, then pipreqs added lib into requirement.txt while this lib acctully refers to https://pypi.org/project/lib/ and confused users that install with this txt file.

I am building an internal proof of concept python monorepo tool and get the same thing but with pkg instead of lib.

At least in my case using the --use-local flag to only use what I have installed locally to build the requirements.txt file without the rogue reference.