conda / conda-lock

Lightweight lockfile for conda environments
https://conda.github.io/conda-lock/
Other
458 stars 102 forks source link

Side-effects from `.condarc`: packages added from `create_default_packages` #492

Open calebzulawski opened 10 months ago

calebzulawski commented 10 months ago

Original title: channels field in .condarc doesn't seem to be respected

Checklist

What happened?

I have a .condarc similar to:

channels:
  - conda-forge
create_default_packages:
  - python

My environment spec doesn't have channels.

When I use conda-lock, I need to add -c conda-forge despite being in my condarc. I know my condarc is getting used because the create_default_packages contents end up in the lockfile.

Conda Info

I'm using micromamba: 

           environment : None (not found)
           env location : -
      user config files : /home/caleb/.mambarc
 populated config files : 
       libmamba version : 1.4.3
     micromamba version : 1.4.3
           curl version : libcurl/7.88.1 OpenSSL/3.1.0 zlib/1.2.13 zstd/1.5.2 libssh2/1.10.0 nghttp2/1.52.0
     libarchive version : libarchive 3.6.2 zlib/1.2.13 bz2lib/1.0.8 libzstd/1.5.2
       virtual packages : __unix=0=0
                          __linux=5.19.0=0
                          __glibc=2.35=0
                          __archspec=1=x86_64
               channels : 
       base environment : /home/caleb/micromamba
               platform : linux-64

### Conda Config

```shell
Configuration files (by precedence order):
./.condarc

### Conda list

```shell
I'm not sure the micromamba equivalent

Additional Context

No response

maresb commented 10 months ago

Thanks a lot for this bug report. This is very interesting.

I'd actually argue that conda-lock picking up the create_default_packages is a bug. The reason is that conda-lock is supposed to transform a specification into a lockfile. Mixing up the specification and the local .condarc config is going to lead to problems and confusion where different computers produce different lockfiles, and the reason being very difficult to track down.

mfisher87 commented 10 months ago

I'd actually argue that conda-lock picking up the create_default_packages is a bug. The reason is that conda-lock is supposed to transform a specification into a lockfile. Mixing up the specification and the local .condarc config is going to lead to problems and confusion where different computers produce different lockfiles, and the reason being very difficult to track down.

:100:

This aspect of conda-lock's reproducibility is IMO really important, but not clearly enough communicated in the README or docs. The docs are really clear that conda-lock's objective is to record enough information so that a conda environment can be created twice identically (excepting packages being removed from repositories out of its control), but not so clear about the advantages of eliminating (excepting changes in repository metadata out of its control) side effects in the solving process itself.

What you said above does communicate that clearly, and it feels like it would be valuable to copy it directly in to the docs :)

calebzulawski commented 10 months ago

Perhaps what I need for my use-case is to transform env.yaml to include channels and default packages, which should have a similar result. I could also see this as similar to environment variables however, where the spec is not complete without knowing the values. I think you could equally include default channel and package information from the .condarc in the lockfile if necessary.

maresb commented 10 months ago

Is there anything about your use case that makes explicitly including everything in the env.yaml cumbersome?

calebzulawski commented 10 months ago

It should be possible--in my use case it happens that the condarc is controlled by a build system and the dependencies are defined by the user, so that's why I care about the distinction.

maresb commented 10 months ago

Ah, got it.

You know, I wouldn't be opposed to adding .condarc as a spec source in the sense of --file=~/.condarc.

calebzulawski commented 10 months ago

I think that would be perfect, assuming you could provide the condarc along with a spec! I actually tried that first before submitting this issue, on a whim that it might work.

maresb commented 10 months ago

I don't have the bandwidth to work on this. Would you like to write a src_parser for .condarc and open a PR?

calebzulawski commented 10 months ago

I'll take a look into it!

maresb commented 10 months ago

Great! And if you see things that seem unnecessarily complicated or lacking in documentation, then if you're feeling up to it please be bold about making changes to clean things up.