Implement explicit lockfiles integration

[jaimergp]

Closes #21

• [X] conda list --explicit extensions
• [x] conda install | create extensions for the above output
• [x] Tests
• [x] Docs

[jaimergp]

This is how the output of conda list --explicit would work, btw:

# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: osx-arm64
@EXPLICIT
https://conda.anaconda.org/conda-forge/osx-arm64/bzip2-1.0.8-h93a5062_5.conda
https://conda.anaconda.org/conda-forge/osx-arm64/c-ares-1.23.0-h93a5062_0.conda
https://conda.anaconda.org/conda-forge/osx-arm64/ca-certificates-2024.2.2-hf0a4a13_0.conda
https://conda.anaconda.org/conda-forge/osx-arm64/icu-73.2-hc8870d7_0.conda
https://conda.anaconda.org/conda-forge/osx-arm64/jpeg-9e-h1a8c8d9_3.conda
https://conda.anaconda.org/conda-forge/osx-arm64/libcxx-16.0.6-h4653b0c_0.conda
https://conda.anaconda.org/conda-forge/osx-arm64/libdeflate-1.17-h1a8c8d9_0.conda
https://conda.anaconda.org/conda-forge/osx-arm64/libedit-3.1.20191231-hc8eb9b7_2.tar.bz2
https://conda.anaconda.org/conda-forge/osx-arm64/libev-4.33-h93a5062_2.conda
https://conda.anaconda.org/conda-forge/osx-arm64/libffi-3.4.2-h3422bc3_5.tar.bz2
https://conda.anaconda.org/conda-forge/osx-arm64/libiconv-1.17-h0d3ecfb_1.conda
https://conda.anaconda.org/conda-forge/osx-arm64/libwebp-base-1.3.2-hb547adb_0.conda
https://conda.anaconda.org/conda-forge/osx-arm64/libzlib-1.2.13-h53f4e23_5.conda
https://conda.anaconda.org/conda-forge/osx-arm64/llvm-openmp-17.0.6-hcd81f8e_0.conda
https://conda.anaconda.org/conda-forge/osx-arm64/lzo-2.10-h642e427_1000.tar.bz2
# Following lines added by conda-pypi
# pypi: gitpython==3.1.32 --python-version 3.9.9 --implementation cp
# pypi: platformdirs==4.2.2 --python-version 3.9.9 --implementation cp
# pypi: sphinx==4.2.0 --python-version 3.9.9 --implementation cp
# pypi: python-dateutil==2.8.1 --python-version 3.9.9 --implementation cp
# pypi: pathspec==0.8.1 --python-version 3.9.9 --implementation cp
# pypi: charset-normalizer==2.0.7 --python-version 3.9.9 --implementation cp
# pypi: tornado==6.1 --python-version 3.9.9 --implementation cp --abi cp39 --platform macosx_11_0_arm64
# pypi: gitdb==4.0.10 --python-version 3.9.9 --implementation cp
# pypi: black==21.6b0 --python-version 3.9.9 --implementation cp
# pypi: regex==2021.4.4 --python-version 3.9.9 --implementation cp --abi cp39 --platform macosx_11_0_arm64
# pypi: tzdata==2023.3 --python-version 3.9.9 --implementation cp
# pypi: pillow==8.3.2 --python-version 3.9.9 --implementation cp --abi cp39 --platform macosx_11_0_arm64

We can't print the URL for PyPI wheels because it wasn't kept around 😬

[jaimergp]

I'm not personally a fan of using the CLI flags here but these are the problems I faced trying to obtain something else:

• There's no record of the origin URL for an installed PyPI package (see https://github.com/pypa/pip/issues/10736). This might change with PEP-710, but it's still a draft.
• The environment markers (`name=version; python_version='2.7' etc) do not limit which wheels will be obtained, but which lines are accepted for inclusion in the request list given the current Python interpreter.

I wish there was a way to encode the constraints in a single string field (like a URI or something), but right now this is the only way, I think. We can document this as the official syntax... which happens to be compatible with a subset of the pip CLI 😬 Note we are adding more flags to the command.

[jaimergp]

Thanks for the thorough review!

I'm not sure how I feel you expanding this format without a spec and a CEP since that's definitely a core functionality in conda

I do want the CEP. For the original format too. There's no standard to refer to other than the implementation. It should be straightforward.

There has to be other ways than implementing custom argv parsing and custom Python packaging standards parsing (big maintenance risk based on my experience)

conda has its own parsers but I only learnt that after writing this prototype. I'm sure we can find some parsers somewhere in packaging and friends too. Right now I'm mostly exploring how this idea would shape up, but I agree we need to polish this custom code better. Even the structure of the code could use some improvement.

Inline docs and high-level reasoning for some of the design decisions are sorely needed before merging this

Yep, just wanted to get some external confirmation that this idea is worth pursuing before writing details on something that might not fly. I take that this review is also a 👍 to proceed further?

[jaimergp]

I do want the CEP. For the original format too.

There you go: https://github.com/conda/ceps/pull/79

[jaimergp]

Hey @jezdez, I think I addressed most of your comments now. The PR is getting a bit too big so we can address further changes in new PRs. WDYT? After this PR I feel comfortable moving things to conda-incubator.

[jezdez]

@jaimergp thank you for doing the rounds on the standardization, this'll be a net benefit!