jaraco / pip-run

pip-run - dynamic dependency loader for Python
MIT License
130 stars 19 forks source link

Pre-install pip-run in the GitHub Actions runner images #94

Open bswck opened 6 months ago

bswck commented 6 months ago

It would be very useful to leverage pip-run as a built-in CLI app immediately available in the vast majority of GitHub Actions. Since pip-run saves people time on having to specify installation steps of Python packages, let's try to remove the necessity to install pip-run anywhere it causes some overhead, too. pipx is already there, what about pip-run?

See also https://github.com/actions/runner-images#preinstallation-policy.

bswck commented 6 months ago

A final impulse to open this issue was caused by two things:

  1. pip-run is not intended to solve production dependency management, but does aim to address the other, one-off scenarios around dependency management: (...) test runners,
  2. I had wanted to primarily use ruff and mypy via pip-run in my CI/CD workflows (and not specify ruff and mypy as non-optional dependencies in my Poetry pyproject.toml configs, because on my PC I prefer using them from my pipx-managed packages), but I abandoned that idea as I had to additionally install pip-run just for that single purpose.

    Well, I could simply pipx install ruff/pipx install mypy prior to using ruff and mypy and I won't need pip-run anymore (note that pipx is pre-installed in GHA runners—a fact previously mentioned in the comment opening this issue).

    The amount of lines stays the same in both cases (1. install pip-run and then pip-run ruff/mypy ... 2. install ruff and mypy via pipx and then use them), so the entire advantage of using pip-run in that specific use case (and probably very common one) is gone...

    ...only because it's not a built-in feature in GHA runners that is ready to use without installation.

bswck commented 6 months ago

On the other hand, I could simply use the caching feature of the setup-python action and not have to install anything except Poetry in my test workflow runtime, cache for which I can have enabled apart from the cachings of pip-managed and pipenv-managed environments.

My workflows are pretty specific, however, and pip-run can be a better approach for CLI apps versions of which I don't intend to track. pip-run might also be especially useful for running local repo scripts in workflows. I even found a way to run a Nox file (without Nox already installed) directly with pip-run noxfile.py.

Moreover, I am pretty sure the overall performance won't be that bad in those "single-use" pipeline runs with the in-memory installation feature I am currently working on.

bswck commented 6 months ago

Related in some way: https://github.com/pypa/pip/issues/3971

jaraco commented 4 months ago

Related in some way: pypa/pip#3971

Yes, my preference would be to have some form of pip-run in pip, as that solves the bootstrapping issue not only for GitHub CI, but for every other Python environment.

Another potential concern - since I last was investing energy into incorporating pip-run into pip, this project has adopted a number of dependencies:

 ~ @ pip-run pipdeptree -- -m pipdeptree -p pip-run
pip-run==12.5.0
├── autocommand [required: Any, installed: 2.2.2]
├── jaraco.context [required: Any, installed: 4.3.0]
├── jaraco.env [required: Any, installed: 1.0.0]
├── jaraco.functools [required: >=3.7, installed: 4.0.0]
│   └── more-itertools [required: Any, installed: 10.2.0]
├── jaraco.text [required: Any, installed: 3.12.0]
│   ├── autocommand [required: Any, installed: 2.2.2]
│   ├── inflect [required: Any, installed: 7.0.0]
│   │   ├── pydantic [required: >=1.9.1, installed: 2.6.0]
│   │   │   ├── annotated-types [required: >=0.4.0, installed: 0.6.0]
│   │   │   ├── pydantic-core [required: ==2.16.1, installed: 2.16.1]
│   │   │   │   └── typing-extensions [required: >=4.6.0,!=4.7.0, installed: 4.9.0]
│   │   │   └── typing-extensions [required: >=4.6.1, installed: 4.9.0]
│   │   └── typing-extensions [required: Any, installed: 4.9.0]
│   ├── jaraco.context [required: >=4.1, installed: 4.3.0]
│   ├── jaraco.functools [required: Any, installed: 4.0.0]
│   │   └── more-itertools [required: Any, installed: 10.2.0]
│   └── more-itertools [required: Any, installed: 10.2.0]
├── more-itertools [required: >=8.3, installed: 10.2.0]
├── packaging [required: Any, installed: 23.2]
├── path [required: >=15.1, installed: 16.9.0]
├── pip [required: >=19.3, installed: 23.3.2]
└── platformdirs [required: Any, installed: 4.2.0]

Note that due to the pydantic dependency, this dependency tree is non-pure, so would not be readily vendored into any package.

I'm exploring ways to eliminate that build requirement on jaraco.text/inflect, but I don't yet have a solution.

Even without the rust build dependency, the large number of dependencies could prove problemmatic for vendoring into pip or for providing as a pre-installed package in a CI environment.