Add support for getting requirements from pyproject.toml based projects.

DavHau / mach-nix

Create highly reproducible python environments

MIT License

862 stars 106 forks source link

Add support for getting requirements from pyproject.toml based projects. #319

Open tomprince opened 3 years ago

tomprince commented 3 years ago

It would be nice if mach-nix supported getting requirements for projects that use pyproject.toml.

I managed to write a wrapper that can extract the requirements here, but given that I use mkPython a couple of times in getting the requirements, I'm not sure how best to integrate the code into mach-nix.

DavHau commented 3 years ago

Thanks that's amazing. For the crawler and mach-nix to use it, it would have to be integrated in: ./lib/extractor/default.nix.

The relevant methods are:

extractor-fast: which is imported and used in the update-sdist flake app of pypy-deps-db
extract_from_src: which is used in buildPythonPackage of mach-nix

Using mach-nix.mkPython to build the environment is fine I think. I don't know of another quick way to generate a valid environment from arbitrary requirements.

tomprince commented 3 years ago

Using mach-nix.mkPython to build the environment is fine I think.

It is not at all obvious how to get access to mkPython from lib/extractor.

tomprince commented 2 years ago

I was revisiting this code recently, and I'm really not sure the how to integrate it.

Using mach-nix.mkPython to build the environment is fine I think. I don't know of another quick way to generate a valid environment from arbitrary requirements.

This would probably be complex but doable for extract_from_src, but I'm not sure how to go about doing it for extractor-fast, since mkPython depends on having access to pypi-deps-db.

tomprince commented 2 years ago

I have a WIP PR at https://github.com/DavHau/mach-nix/pull/380 that replaces the uses of extract_from_src that works in some simple cases. It doesn't handle packages with extras, for example, and appears to need a bunch of packages specified as provided by wheels (probably because pypi-deps-db is lacking proper dependencies for them).