dynamically generated PIP_EXTRA_INDEX_URL before installing.

[gerbendekkervolue]

We host some packages on a private package repo in GCP. To auth to this I would typically run:

ACCESS_TOKEN=$(gcloud auth print-access-token)
export PIP_EXTRA_INDEX_URL=https://oauth2accesstoken:$ACCESS_TOKEN@europe-python.pkg.dev/my_company/simple/

Now, I would like to configure hatch through pyproject.toml to run this and keep this PIP_EXTRA_INDEX_URL in two cases:

1. before installing the hatch default env
2. when running selected hatch scripts (using pip-compile to lock dependencies and export them to a requirements.txt file).

It is in a way similar to this, https://hatch.pypa.io/latest/how-to/environment/package-indices/, apart from the access token to be generated on the fly.

Is this possible to achieve?

When just putting the two lines above in a pre-install-commands block, it seems the environment variables are not persisting.

[zeyus]

I have a very similar problem.

I'm trying to make a package to support using pytorch with either CPU or GPU on windows, mac and linux. The way pytorch installs, it requireds a PIP_INDEX_URL for windows GPU and linux CPU...but I can't for the life of me figure out a way to get this to work.

I've tried using the plugin hatch-requirements-txt but it also doesn't support pip flags or options. Further, there doesn't seem to be any way for a metadata plugin hook to access or modify the environment variables. Nor is it possible to modify the command sent to pip (there's no clear path for injection without modifying the core hatch source).

pyproject.toml:

[build-system]
requires = ["hatchling", "hatch-requirements-txt"]
build-backend = "hatchling.build"

[project]
# ...
dynamic = ["version", "dependencies", "optional-dependencies"]
# ...

[tool.hatch.metadata.hooks.requirements_txt]
files = ["requirements.txt"]

[tool.hatch.metadata.hooks.requirements_txt.optional-dependencies]
cpu = ["requirements-cpu.txt"]
gpu = ["requirements-gpu.txt"]

# ...

[tool.hatch.envs.default.env-vars]
# PIP_INDEX_URL = "https://download.pytorch.org/whl/nightly/cu121" # windows gpu
PIP_INDEX_URL = "" # windows cpu, linux gpu, all mac
# PIP_INDEX_URL = "https://download.pytorch.org/whl/cpu" # linux cpu
PIP_PRE = "1"

This seems like a really useful feature to have...either

1. Allow specification of environment variables that change based on optional dependencies; or
2. Allow per-dependency pip settings
3. Plugins 3.1. Allow plugins / hooks access to environment variables 3.2. Allow plugins / hooks to modify the pip install command

[johnpyp]

You can use an Environment Collector plugin for this - I've made a very simple one for a nearly identical usecase of editing environment variables for every env, including the PIP_INDEX_URL.