search

con / tinuum

Resources (later might be a tool) to create reproducible computational environments
Apache License 2.0
0 stars 0 forks source link

PyPI (pip) #1

Open yarikoptic opened 5 months ago

yarikoptic commented 5 months ago

https://pypi.org/project/pypi-timemachine/

matrss commented 1 month ago

The pixi package manager has support for using packages from PyPI in addition to conda channels now. See e.g. here for more info: https://prefix.dev/blog/using_python_projects_with_pixi

This makes it possible to have one unified lock file for your dependencies from both PyPI and e.g. conda-forge. So it could support both #1 and #2.

yarikoptic commented 1 month ago

@matrss thanks for the pointer! Do you know is there is some way to pin to a specific state/date of the entire pypi distribution? (To a degree that's what time machine would achieve)

matrss commented 1 month ago

Sort of, but not exactly.

The workflow with pixi looks like this:

  1. pixi init a directory to turn it into a pixi project
  2. pixi add your dependencies, e.g. to get python and matplotlib from PyPI: pixi add python; pixi add --pypi matplotlib
  3. pixi shell to start a shell with the installed packages present, or pixi run <command> to execute a command in that environment

This will produce a pixi.toml file like this:

$ cat pixi.toml 
[project]
authors = ["Matthias Riße <m.risse@fz-juelich.de>"]
channels = ["conda-forge"]
description = "Add a short description here"
name = "pixi-demo"
platforms = ["linux-64"]
version = "0.1.0"

[tasks]

[dependencies]
python = ">=3.12.5,<4"

[pypi-dependencies]
matplotlib = ">=3.9.2, <4"

which list a bit of metadata about the project and the dependencies that were added. At the same time, the pixi add commands will create an accompanying pixi.lock file, which describes the solved environment.

Now, as long as pixi.lock is not modified, all future invocations of pixi shell or pixi run will use the same environment. This file can be checked into version control and shared with others. To get an updated environment with the version constraints specified in pixi.toml the lock file must be recreated.

This isn't exactly what you were asking for, but I'd imagine it solves the same issue for which you would want to pin a state of PyPI. Does it?

matrss commented 1 month ago

Ah, I just realized that pypi-timemachine behaves as an alternative server to point pip to. I'd imagine you could also just point pixi to that: https://pixi.sh/v0.27.1/reference/project_configuration/#alternative-registries. But again, I suspect that most issues this would solve are already addressed by the existence of the lock file.