Replace pipenv with a better dependency management tool

[hogo6002]

OSV.dev currently uses Pipenv for managing Python dependencies, but it's causing a number of issues, such as https://github.com/google/osv.dev/pull/2170.

Key Concerns with Pipenv PyPA Recommendation: Pipenv is not officially recommended by the PyPA. They suggest tools like pip, venv, Poetry, or PDM. Design issues: Pipenv's design relies on internal components of pip and venv, which may lead to compatibility issues and unexpected behavior. Negative Impact on OSV.dev: These issues increase maintenance work and sometimes cause problems with our automated dependency updates.

Proposed Alternatives Bazel? Poetry? PDM?

[G-Rath]

fwiw we've moved everything at Ackama to use Poetry, though we were previously primarily using pip / pip-compile

[andrewpollock]

/cc @di for input

[another-rex]

@G-Rath pip-compile is one of the choices we are thinking of going with, any particular reason you folks moved to Poetry?

[G-Rath]

It was primarily that we found Poetry tends to mirror a lot of the other dependency managers we're using like npm, composer, bundler, making us feel better about managing dependencies with it e.g. we can do "poetry update " to get a security update for some nested child package without having everything else upgraded or having to add that package as a direct dependency, and all the packages are captured in the single predictably-placed file that is poetry.lock regardless of what group they're in (rather than requirements.dev and what have you)

FWIW: Poetry's resolution strategy doesn't exactly match pip's, meaning that the two shouldn't be assumed to be always consistent/identical for all resolutions. This may or may not matter, depending on your deployment environment/conditions 🙂

In terms of consistency with the larger ecosystem (and PyPA in particular), my recommendation would be pip-compile: it uses pip's own resolver, and would mirror what PyPI and other large codebases use to keep their Python dependencies fully pinned.

[github-actions[bot]]

This issue has not had any activity for 60 days and will be automatically closed in two weeks

See https://github.com/google/osv.dev/blob/master/CONTRIBUTING.md for how to contribute a PR if you're interested in helping out.

[andrewpollock]

@another-rex you're actively working on this, right? Want to assign it yourself so it doesn't go stale?