Configure private PyPi repositories in the environment

[jacksmith15]

Description

Resolves #460

This PR adds support for configuring private PyPi repositories via environment variables. This avoids users needing to install an exact version of poetry to manage the configuration format, and then copy around credential files to make them discoverable to conda-lock. See #460 for full reasoning.

After this change, users are able to set the CONDA_LOCK_PYPI_REPOSITORY_<repository-name> environment variable to configure new PyPi repositories.

An alternative could be to support PIP_EXTRA_INDEX_URL instead, as this will be respected during conda-lock install, meaning only a single environment variable is required for both lock and install.

[netlify[bot]]

✅ Deploy Preview for conda-lock ready!

Name	Link
🔨 Latest commit	665e8c859051048001a21fc1337b8fae93615e9d
🔍 Latest deploy log	https://app.netlify.com/sites/conda-lock/deploys/64ff2091c5082e0008082e56
😎 Deploy Preview	https://deploy-preview-471--conda-lock.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[jacksmith15]

@maresb thanks for the review, and apologies for the delay in updating.

I think I've addressed all of your previous feedback now :+1:

[maresb]

Hey @jacksmith15, I feel really bad for doing this, but after taking a fresh look I think we are on the wrong track here. :cry: I should have caught this earlier in the review process, my bad.

I think a much better approach would be a parallel implementation of authenticated channels where the corresponding documentation would be essentially identical, but substituting channels → pypi-repositories.

Do you agree? If so, would you have any time to rework this into a (partial) implementation?

I'm terribly sorry about sending you down the wrong track. Please let me know if I can do anything to help.

[jacksmith15]

To be clear, the suggested approach would involve adding a new non-standard key to the source environment.yml, e.g.

channels:
  - https://host.tld/t/$QUETZ_API_KEY/channel_name
  - conda-forge
repositories:
  - https://$MY_USER:$MY_PASS@example.repo/pypi/simple

Does that look right to you? I note this doesn't support naming repositories, which is how they are generally supplied to the poetry solver. Should they be auto-named, or break the structural convention in the file?

Should this also provide a mechanism for enabling/disabling PyPi? Similar to nodefaults in channels?

Additionally, should --strip-auth apply to env var references?

[maresb]

These are very good questions. You are understanding my proposal correctly.

We already have a precedent for defining a non-standard key in environment.yml, namely platforms, so I think adding another should be okay.

I personally would prefer to be more specific with the key name and have it be pypi-repositories just because "repository" is such a general term, and I get confused wondering what's Conda-related and what's pip-related. Maybe it should be called pip-repositories? (Note - characters should be fine for key names in yaml, toml, and command-line arguments.)

I think it's important to avoid Poetry's influence which may push us towards bad design decisions, so I'd prefer autonaming to having a discrepancy between channels and repositories. Do you foresee any problems that might arise with this approach?

A mechanism for disabling PyPI outside of pyproject.toml and more in line with this system would be interesting, but certainly not necessary at this stage. Probably it would be best to defer this to another PR.

Supporting --strip-auth would be nice, but if it's a lot of work I think it'd be fine to instead add a prominent disclaimer.

[jacksmith15]

@maresb regarding the naming, its actually a little tricky to give a good name for Python repositories. I think the most "correct" term is "python simple repository": https://packaging.python.org/en/latest/specifications/simple-repository-api/#simple-repository-api

• pypi refers specifically to https://pypi.org/
• pip refers to the package manager, which is one way to interact with Python Package repositories which conform to the Simple API

However simple-repositories has a similar issue of being quite vague! I suppose one could do pep-503-repositories but that requires the reader to have additional context.

I'll prefer pip-repositories for now as that's probably most familiar to most people, but worth considering which you prefer.

[maresb]

@jacksmith15, I agree that I think pip-repositories is the best choice.

[jacksmith15]

:memo: I haven't forgotten about this!

I am working on the alternative solution in https://github.com/conda/conda-lock/pull/481

There's some differences with pip repositories compared to channels that need handling, but I the general direction seems :+1:

[maresb]

Superseded by #529