ido50 / morgan

PyPI Mirror for Restricted/Offline Environments
Apache License 2.0
105 stars 7 forks source link

Add option to generate requirements from current env #1

Closed Yozh2 closed 2 years ago

Yozh2 commented 2 years ago

This pull-request adds option to automatically generate requirements configuration block using packages from current python environment. The requirements block is printed to standard output, and can either be copied to the configuration file, or piped to it using shell redirection (e.g. >>). The packages are sorted alphabetically.

$ morgan generate_reqs
[requirements]
anyio = >=3.6.1
appdirs = >=1.4.4
appnope = >=0.1.2
apptools = >=5.2.0
...

The --mode argument is used for precise versioning: ">=" / "==" / "<="

$ morgan generate_reqs -m "=="
[requirements]
anyio = ==3.6.1
appdirs = ==1.4.4
appnope = ==0.1.2
apptools = ==5.2.0
...

This PR would be handy, if you want to maintain (and update regularly) a PYPI mirror with packages similar to your work environment on your online machine.

ido50 commented 2 years ago

Hi @Yozh2, thanks a lot for your contribution. This looks like a great utility. I actually thought of something similar that generates requirements from a project file such as pyproject.toml or requirements.txt, which can probably be built on top of your new function.

I have one problem though: pkg_resources is part of setuptools, which isn't currently a requirement of morgan. On the one hand, I'm not sure I want to make it a requirement of the project. On the other hand, it's installed on basically any environment out there, so maybe it doesn't matter. Regardless, you'll need to add it as a requirement in pyproject.toml (via poetry add setuptools).

Yozh2 commented 2 years ago

@ido50, thank you for a quick reply.

I tried several approaches to get a list of installed packages. One of them included manual parsing stdout from subprocess.run("pip list"), but it seemed too gross, bug-prone and non-pythonic to me. I also tried to use pip internal functions, but that approach failed because the devs removed this option and strongly advised not to use the internal functions of pip.

I agree, that setuptools is installed by default in almost any distribution nowadays. Actually, the pip devs themselves recommend using pkg_resources to get a list of installed packages.

If you really don't want to add setuptools to the list of required dependencies, I can rewrite the code using the first method (subprocessing the pip list command and parsing the output). It may be beneficial, as you want to add support for parsing the requirements.txt file.

ido50 commented 2 years ago

@Yozh2, note that I've released version 0.10.0 with this new feature. I ended up replacing pkg_resources with importlib.metadata, as I noticed on their official documentation that usage is discouraged in favor of importlib.metadata. I verified the output is the same.

Thanks a lot!