Request: leave environment variable references as-is in requirements.txt output.

[metavee]

What's the problem this feature will solve?

When using private package indexes, you can supply credentials in the index URL via environment variables.

--extra-index-url https://${PRIVATE_PYPI_USER}:${PRIVATE_PYPI_PASSWORD}@my-private-pypi-index/...

If you do this in your requirements.in file, pip-compile will output the credentials into the requirements.txt file.

This makes it difficult to use pip-tools when using a private index, since ideally you could commit requirements.txt to version control without having to make any tweaks or downstream adjustments.

Describe the solution you'd like

I would like any environment variable references to appear in the output requirements.txt file, without the actual values being substituted in.

Alternative Solutions

Additional context

This could complement https://github.com/jazzband/pip-tools/issues/1876 or make it unnecessary.

[webknjaz]

Does pip itself support references to env vars in requirements files? I don't recall seeing it there...

[metavee]

Yes, it does.

https://pip.pypa.io/en/stable/reference/requirements-file-format/#using-environment-variables

[webknjaz]

Ah, interesting. Makes sense, then. Though, personally, I'd want the constraint files to be generated without any extra options.

[metavee]

Is there an easy alternative?

I know pip-compile will pass through pip arguments such as the index URL, but then that shifts the credentials to the pre-commit config, which is also in version control and doesn't natively support environment variable substitutions.

[webknjaz]

I'm not sure. Does pip support ~/.netrc?

[WhyNotHugo]

It does: https://stackoverflow.com/a/55978589/107510

machine pypi.example.com
    login johndoe
    password changeme

[WhyNotHugo]

As per the above link, it also seems to be possible to use environment variables in pip.conf.

[webknjaz]

Case closed, then?

[metavee]

Yeah, I guess so! It seems like there is a reasonably safe way to use custom indexes without revealing secrets in the files.

I think there is still a minor footgun since it's surprising, but maybe #1876 is the easy way to address that.

[webknjaz]

Yep, that sounds like a reasonable request.