readthedocs / readthedocs-docker-images

Docker image definitions used by Read the Docs
115 stars 70 forks source link

[Featre request] Include `pipx` with proper configuration #208

Open huxuan opened 4 weeks ago

huxuan commented 4 weeks ago

Though we already have asdf, it might be better to also have pipx so that we can install Python Applications directly from PyPI when needed.

Current workaround in .readthedocs.yaml:

      - asdf plugin add pipx
      - asdf install pipx latest
      - asdf global pipx latest
      - PIPX_BIN_DIR=$READTHEDOCS_VIRTUALENV_PATH/bin pipx install <package>

The mostly inconvenience is that we have to specify the PIPX_BIN_DIR to make it actually work. It would be great if we can have it properly configured just like asdf.

humitos commented 3 weeks ago

Hi @huxuan! Thanks for opening this issue. Can you expand a little more on this use case? In particular, what's the different between running pipx and just the regular pip to install packages?

huxuan commented 3 weeks ago

Can you expand a little more on this use case? In particular, what's the different between running pipx and just the regular pip to install packages?

Generally if possible, I would like to install those standalone tools in isolated virtual environments. And currently, there are at least three of them:

  1. pdm: manage and install plugins and theme for building documentation
  2. mypy: generate mypy report and include them in the generated documentation in HTML format
  3. coverage: same as mypy but generating coverage report

All of these tools have nothing to do with building documentation, and installing them with pip will introduce a lot of redundant dependencies and increase the possibility of dependency conflicts and build failures.

stsewd commented 3 weeks ago

Looks like you can also install pipx from apt https://pipx.pypa.io/latest/installation/ (https://docs.readthedocs.io/en/stable/config-file/v2.html#build-apt-packages).

huxuan commented 3 weeks ago

Looks like you can also install pipx from apt https://pipx.pypa.io/latest/installation/ (https://docs.readthedocs.io/en/stable/config-file/v2.html#build-apt-packages).

Wow, that is cool! I did not notice that configuration. Even though that does not ensure to use the latest version of pipx, it should be a better solution than the workaround I mentioned in the issue description. Thanks!

huxuan commented 3 weeks ago

Hi, sorry for heads up again.

TL;DR;

Can we add /home/docs/.local/bin in PATH here? https://github.com/readthedocs/readthedocs-docker-images/blob/4d356fa1586f74bd39c56c4789f383686f1d7ed9/Dockerfile#L137

I can send a pull request if it is OK.


I tried to use the build.apt_packages, it still suffers on the following PATH issue. And you can refer to the full build log here: https://readthedocs.org/projects/ss-python/builds/24818740/

⚠️ Note: '/home/docs/.local/bin' is not on your PATH environment variable.

One solution could be running pipx ensurepath and source ~/.bashrc explicitly. Otherwise, we have to specify the PIPX_BIN_DIR when installing packages, do you have any ideas about better solutions?

The currently workaround I use is as following which is already better than the previous version. :-)

build:
  apt_packages:
    - pipx
  jobs:
    pre_create_environment:
      - PIPX_BIN_DIR=$READTHEDOCS_VIRTUALENV_PATH/bin pipx install mypy[reports]==1.10.1
      - PIPX_BIN_DIR=$READTHEDOCS_VIRTUALENV_PATH/bin pipx install pdm==2.15.4
humitos commented 3 weeks ago

@huxuan can you try running asdf reshim python after asdf global pipx latest? I think that should remove the need for defining PIPX_BIN_DIR environment variable.

huxuan commented 3 weeks ago

@huxuan can you try running asdf reshim python after asdf global pipx latest? I think that should remove the need for defining PIPX_BIN_DIR environment variable.

Currently, I want to avoid using asdf, the build.apt_packages works quite well except for the PATH issue.