gezabohus commented 1 year ago

Hi all,

This is on MacOS Catalina. My project uses python 3.10, but I also have python 3.8 installed as well as python 2.7 as it is used by the system.

I run

pipreqs .

in the project directory and for every import it finds in the code issues

"WARNING: Import named "..." not found locally. Trying to resolve it at the PyPI server."

even though these are in fact installed for python 3.10.

How do I make it find them?

Thanks.

waivek commented 1 year ago

I'm having the same issue. I'm on WIndows.

k1mbl3 commented 1 year ago

Same here on linux

jamescl604 commented 1 year ago

Same here, also on Windows

From me, it's scipy and it looks like the pipreqs code identifies local packages by walking your sys,path and finding packages that have a dist-info/top-level.txt file. Scipy does not (on my machine) and Numpy does, which is why I get Numpy listed but not Scipy.

waltk-ricoh commented 1 year ago

I'm having the same problem running pipreqs against a package containing a flask app on Linux. pipreqs seems to be telling me that the flask app is using dozens of packages that are not actually present on the system. That is a real concern for me and, if it is really true, would prevent my using flask for anything. I have to have local copies of what I use, because it is a corporate environment. It is not acceptable to just download random python packages off of the internet everytime you run an enterprise app. Perhaps this is needless to say, but there: I said it.

DanielYang59 commented 1 year ago

The same issue in MacOS Sonoma 14.0. Python3 managed by Anaconda3.

$ pipreqs --print
WARNING: Import named "numpy" not found locally. Trying to resolve it at the PyPI server.
WARNING: Import named "numpy" was resolved to "numpy:1.26.0" package (https://pypi.org/project/numpy/).
Please, verify manually the final list of requirements.txt to avoid possible dependency confusions.
WARNING: Import named "pandas" not found locally. Trying to resolve it at the PyPI server.
WARNING: Import named "pandas" was resolved to "pandas:2.1.1" package (https://pypi.org/project/pandas/).
Please, verify manually the final list of requirements.txt to avoid possible dependency confusions.
WARNING: Import named "PyYAML" not found locally. Trying to resolve it at the PyPI server.
WARNING: Import named "PyYAML" was resolved to "PyYAML:6.0.1" package (https://pypi.org/project/PyYAML/).
Please, verify manually the final list of requirements.txt to avoid possible dependency confusions.
ase==3.22.1
matplotlib==3.8.0
numpy==1.26.0
pandas==2.1.1
pymatgen==2023.9.10
pytest==7.4.2
PyYAML==6.0.1
PyYAML==6.0.1
tqdm==4.66.1
INFO: Successfully output requirements

Meanwhile:

$ uname -a
Darwin Yang-MacMini 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:42:57 PDT 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T8112 arm64

jrwds commented 1 year ago

352 is similar issue.

Having this issue with Scipy and Seaborn. They are installed locally, but not detected. Causes a secondary Numpy to attempt to be installed, which conflicts with the already requested Numpy version.

WARNING: Import named "scipy" not found locally. Trying to resolve it at the PyPI server. WARNING: Import named "scipy" was resolved to "scipy:1.11.3" package (https://pypi.org/project/scipy/). Please, verify manually the final list of requirements.txt to avoid possible dependency confusions. WARNING: Import named "seaborn" not found locally. Trying to resolve it at the PyPI server. WARNING: Import named "seaborn" was resolved to "seaborn:0.13.0" package (https://pypi.org/project/seaborn/).

Local pip freeze output shows scipy==1.9.3 and seaborn==0.12.2 and numpy==1.23.5

2.259 ERROR: Cannot install numpy==1.23.5 and numpy==1.24.2 because these package versions have conflicting dependencies. 2.260 2.260 The conflict is caused by: 2.260 The user requested numpy==1.23.5 2.260 The user requested numpy==1.24.2 2.260 2.260 To fix this you could try to: 2.260 1. loosen the range of package versions you've specified 2.260 2. remove package versions to allow pip attempt to solve the dependency conflict 2.260 2.261 ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

Generated requirements.txt file: matplotlib==3.6.2 numpy==1.23.5 numpy==1.24.2 pandas==1.5.3 scipy==1.11.3 seaborn==0.13.0

Edit: formatting

jrwds commented 1 year ago

I resolved the issue I had. Turns out it was two separate issues. ~~Here is my temporary fix (use with caution):~~

~~1. Search at root level for dist-info. This will list all of your package folders.~~ ~~2. Find the packages which are being copied more than once. In my case, I had multiple versions of numpy, including 1.23.5 and 1.24.2. Sure enough, I found dist-info packages for both.~~ ~~3. Delete the one which is causing problems~~

~~This isn't a good long term fix, because you might have many environments/installations.~~

I was using a bash shell that was accessing the wrong virtual environment, as I've many environments. Using

source /path/to/environment

inside the Bash script that is using pipreqs solved this issue, as sys.path now looks in the correct location for packages.

The second issue was seaborn and scipy not being found. Again, temporary fix, but I resolved this by adding the top_level.txt file and writing the name of the package in it. It now finds the local installations correctly.

I think the solutions to pipreqs code to account for these issues might be:

don't look for top_level.txt; it's not a guaranteed find, although its in just about every package.
~~Instead, search in the user configured environment folder~~ Already done if you switch the environments using source
Search for dist-info in the environment folder, and then walk over all the directories using os.walk, retrieving the names of packages and versions by parsing -version.number.0, using "-" to split

~~These should be a long term fix that doesn't require the two temp fixes above.~~ I believe the solution will be to remove if "top_level" in item: in get_locally_installed_packages(), seems someone already has had the same idea to split based on the dash. I will submit proposed changes that should dynamically read the name without using top_level.txt.

jrwds commented 1 year ago

I've written a replacement that I use locally. If others are interested, I can create a pull request for it. This assumes there is only one virtual environment to be found in sys.path, which is the case for me but I am unsure if that's true always, since sys.path can be edited. You will also have to ctrl-f for 'exports' and remove those lines of code as well, I am not sure if exports is really necessary anymore.

def get_locally_installed_packages(encoding=None):
    packages = []

   #for each possible path,
    for path in sys.path:

        #if site-packages is in the path name
        if "site-packages" in path:

            #for each file in the site-packages directory,
            for file in os.listdir(path):
                #if it contains dist-info in the directory name, it will contain the version info
                if "dist-info" in file:

                    #parsing out the package name and version from 
                    package_name = file.split(os.sep)[-1].split("-")
                    package = package_name[0]
                    version = package_name[1]
                    version = version.split(os.sep)[-1].split(".dist")
                    version = version[0]

                    #appending the package and version
                    packages.append({
                        'name': package,
                        'version': version
                    })

    return packages

Nugine commented 10 months ago

Any updates? I'm using a workaround script to replace pipreqs in my small projects.

jjaychen1e commented 7 months ago

same to me.

Moonmore commented 5 months ago

I solved this problem. The cause of this problem is caused by the python environment. Similarly, pipreqs is installed in /path/to/pipreqs, but after switching to the virtual environment or conda environment, the pipreqs insert are still called under the original environment.

The solution is to install pipreqs in the current environment:

pip install pipreqs && which pipreqs. Generate requirements using pipreqs in the current environment.

gezabohus commented 5 months ago

Thanks @Moonmore for the comment.

I was not using an environment. Maybe I should.

Moonmore commented 5 months ago

Thanks @Moonmore for the comment.

I was not using an environment. Maybe I should.

If you don't have an environment to use, but have multiple python versions. Then there will be a problem with pipreqs finding multiple versions of installation packages. So you only need to specify a python version, use the corresponding pip to install pipreqs, and then generate it. The reason is that multiple pythons will have multiple versions of installation packages, and pipreqs will issue warnings when encountering multiple versions of installation packages. To put it simply, the solution is that pipreqs only searches for the installation package under the current python, Whether it is python environment or python version.

phgas commented 3 months ago

Same issue here, was anyone able to solve it? Uninstalled pipreqs from local envs and only installed it on (base) conda. Error is still occurring...

gray2bgr commented 1 month ago

I solve it by pipreqs . --use-local

bndr / pipreqs

pipreqs says "WARNING: Import named "..." not found locally. Trying to resolve it at the PyPI server." #374

352 is similar issue.