conda / conda-lock

Lightweight lockfile for conda environments
https://conda.github.io/conda-lock/
Other
458 stars 102 forks source link

Add support PyPi packages when using `--strip-auth` for `--kind lock` lockfiles #461

Open jacksmith15 opened 11 months ago

jacksmith15 commented 11 months ago

Checklist

What is the idea?

https://github.com/conda/conda-lock/pull/323 added support for stripping credentials from PyPi packages in --kind explicit lockfiles.

I propose adding support for --kind lock lockfiles too, since these are the default.

Why is this needed?

Currently running conda-lock lock --strip-auth when private PyPi repositories are configured results in credentials being written directly to the lockfile, meaning it cannot be committed to source control.

Currently this blocks users who have the following constraints

What should happen?

A biasic solution would be to allow stripping auth in --kind lock lockfiles. E.g. conda-lock lock --strip-auth --kind lock should strip credentials from private PyPi repositories urls.

HoweverI think a more flexible solution would be to support transparent environment variable references in configured private PyPi repositories. In this solution the workflow would be something like:

  1. Configure PyPi repositories with environment variable references (requires an additional configuration layer in conda-lock, see #460)
  2. Environment variables are resolved during conda-lock lock step for the resolver
  3. URLs written to the lockfile contain un-resolved environment variable references, e.g.
    - name: private-package
      version: 1.0.0
      manager: pip
      platform: linux-64
      dependencies:
        requests: '*'
      url: https://${PYPI_USER}:${PYPI_PASSWORD}@custom.repo/api/pypi/pypi/private-package/1.0.0/private_package-1.0.0-py3-none-any.whl
      hash:
        sha256: 64c75025fc723c1c29c9c60ad709b639251a896a0c7715c3f5acf00242b48644
      category: main
      optional: false
  4. conda-lock install supports dereferencing these environment variables at install time.

I'm not certain about the complexities involved in doing this, but if there was support for the general idea I would happily attempt an implementation.

Additional Context

Note also https://github.com/conda/conda-lock/issues/460 which proposes improvements to how auth is configured for private PyPi repositories.