pythonlib: improved dependency management and examples

jodersky commented 5 days ago

Make the dependency management more flexible:

[x] allow users to re-use requirements.txt files (inspired by the same pants feature)
[x] separate library and dev requirements
- library deps are transitive, and will in the future be included as requirements in distributions
- dev requirements are not transitive and won't be included in distributions. They're used for things like test runners, type checkers and bundlers (which can all be extracted into separated traits later on if we want to).
[x] ~support referencing wheel files directly~ this can already be done by declaring a direct reference in pythonDeps (pants also supports this)
[x] allow user-defined PYTHONPATH
[x] custom package repos
[ ] ~lockfile support~ Removing this from this pull request, since it's likely a bit more complex than I initially thought. Pants delegates this behavior to PEX, but we should see if this can be done with more basic tools, or we decide to rely on pex for everything.
[x] examples for all the above

Part of #3928

himanshumahajan138 commented 4 days ago

@jodersky Sir I have done the Basic example and if you want/allow i can help in writing examples for this also...

jodersky commented 4 days ago

thanks, however I already started working on the examples

com-lihaoyi / mill