conda-lock creates env in env subdirectory when called from an env that is not the conda base env

[Titus-von-Koeller]

Thanks again for your work, we really appreciate it!

Running the command conda run -n mlops conda-lock install -n uberlog-ai uberlog-ai.conda-lock.yml, I get the unexpected result of my-env being installed in a subdirectory of the mlops-env which is home to my conda-lock installation. It seems conda-lock assumes that it is always used from the base env. This doesn't make sense in our use-case.

We're using conda-lock version 1.0.5: https://conda.anaconda.org/conda-forge/noarch/conda-lock-1.0.5-pyhd8ed1ab_1.tar.bz2

Observed behavior:

$ conda run -n mlops conda-lock install -n my-env my-env.conda-lock.yml
$ conda env list
# conda environments:
#
base                  *  /home/nelius/miniconda3
mlops                    /home/nelius/miniconda3/envs/mlops
                         /home/nelius/miniconda3/envs/mlops/envs/my-env

Excpected behavior:

$ conda run -n mlops conda-lock install -n my-env my-env.conda-lock.yml
$ conda env list
# conda environments:
#
base                  *  /home/nelius/miniconda3
mlops                    /home/nelius/miniconda3/envs/mlops
my-env                   /home/nelius/miniconda3/envs/my-env

[mariusvniekerk]

Yeah this is more a result of how conda works. If you have a conda executable in an environment it effectively becomes a root environment as well.

I would recommend you install conda-lock into your base environment

$ mamba install -n base -c conda-forge conda-lock
# use the subcommand expansion present in conda to find the conda-lock executable
$ conda lock install -n my-env my-env.conda-lock.yml 

or to run the install using

$ conda run -n mlops conda-lock install -p /some/fully-qualified-path my-env.conda-lock.yml

[mariusvniekerk]

The other thing that may work is to have a configuration like

envs_dirs:
  - /home/nelius/miniconda3/envs/

in your ~/.condarc

[Titus-von-Koeller]

Hi, wow, such a fast reply @mariusvniekerk, thanks!

Yeah, yesterday after I raised this issue and before reading your answer, I actually came up with a --prefix solution as well. There is a major blocker there though, see below.

Regarding the base env: Unfortunately, that doesn't work for our use case. Basically we've developed an mlops-CLI that we use to streamline/automate/standardize or MLOps workflows while working with a package alongside: The actual machine learning code. The MLOps tool instantiates its dependencies reproducibly via lockfile and the mlops env -- which wouldn't be possible with the base env.

The funny thing is that I just wanted to say "A bit like pipx and remembered the condax project that I had stumbled across recently: I just looked condax up again and was funnily surprised to see that it is on your personal Github :D, so that's your brainchild... Cool project!

Strangely, everything had seemed to be working fine, but something is still screwy and it seems to be related to how

$ /home/nelius/miniconda3/bin/conda create -n $(MLOPS_ENV) 'conda-lock=1.*' -y   # bootstrap environ
$ /home/nelius/miniconda3/bin/conda activate mlops
$ /home/nelius/miniconda3/bin/conda run --prefix /home/nelius/miniconda3/envs/mlops conda-lock install --dev --prefix /home/nelius/miniconda3/envs/mlops mlops.conda-lock.yml 

INFO:root:Preparing transaction: ...working... done                                                                                                                                                                                                            
INFO:root:Verifying transaction: ...working... done                                                                            
INFO:root:Executing transaction: ...working... done   

$ /home/nelius/miniconda3/bin/conda run --prefix /home/nelius/miniconda3/envs/mlops conda-lock install --dev --prefix 
/home/nelius/miniconda3/envs/mlops mlops.conda-lock.yml

[.. mamba logo ...]
ERROR:root:
ERROR:root:CondaValueError: The target prefix is the base prefix. Aborting.
ERROR:root:
Traceback (most recent call last):
  File "/home/nelius/miniconda3/envs/mlops/bin/conda-lock", line 10, in <module>
    sys.exit(main())
  File "/home/nelius/miniconda3/envs/mlops/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/nelius/miniconda3/envs/mlops/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/nelius/miniconda3/envs/mlops/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/nelius/miniconda3/envs/mlops/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/nelius/miniconda3/envs/mlops/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/nelius/miniconda3/envs/mlops/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/nelius/miniconda3/envs/mlops/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 1268, in install
    install_func(file=lockfile)
  File "/home/nelius/miniconda3/envs/mlops/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 205, in do_conda_install
    _conda(
  File "/home/nelius/miniconda3/envs/mlops/lib/python3.10/site-packages/conda_lock/invoke_conda.py", line 127, in _invoke_conda
    raise subprocess.CalledProcessError(
poetry.utils._compat.CalledProcessError: Command '['/home/nelius/miniconda3/envs/mlops/bin/mamba', 'create', '--file', '/tmp/tmphj_rs9zz', '--yes', '--prefix', '/home/nelius/miniconda3/envs/mlops']' returned non-zero exit status 1.

ERROR conda.cli.main_run:execute(49): `conda run conda-lock install --dev --prefix /home/nelius/miniconda3/envs/mlops mlops.conda-lock.yml` failed. (See above for error)

This seems to be related to:

/home/nelius/miniconda3/envs/mlops/lib/python3.10/site-packages/mamba/mamba.py
610 def create(args, parser):                                                       
611     if is_conda_environment(context.target_prefix):                             
612         if paths_equal(context.target_prefix, context.root_prefix):             
613             raise CondaValueError("The target prefix is the base prefix. Aborting.")

/home/nelius/miniconda3/envs/mlops/lib/python3.10/site-packages/conda/base/context.py
 609     @memoizedproperty                                                           
 610     def root_prefix(self):                                                      
 611         if self._root_prefix:                                                   
 612             return abspath(expanduser(self._root_prefix))                       
 613         elif conda_in_private_env():                                            
 614             return abspath(join(self.conda_prefix, '..', '..'))                 
 615         else:                                                                   
 616             return self.conda_prefix  

At this point, I'm a bit out of my depth, I cannot really follow how the root_prefix get's resolved and if there's any way to hack around this as a temporary fix, as this is a major blocker for me right now. I would be willing to try and contribute a fix longterm (if possible, as the issue seems in both cases Conda and not Conda-lock), but right now I just need to get this working...

P.S. the ~/.condarc entry didn't help either unfortunately.

Do you have any further ideas on this?

[Titus-von-Koeller]

To summarize, the weird thing in the previous stack trace is that after bootstrapping the env to include conda-lock, the first conda-lock install actually goes through successfully. Then, the second fails in this strange way..

[Titus-von-Koeller]

To be even more specific, I would have expected this operation to be idempotent.

[Titus-von-Koeller]

Is there an efficient way to check if an env is still in line with it's lock file, based on some hashing for example?

[mariusvniekerk]

you might be able to get this to work by passing the path to conda explicitly

$ /home/nelius/miniconda3/bin/conda run --prefix /home/nelius/miniconda3/envs/mlops \
    conda-lock install --dev \
    --prefix /home/nelius/miniconda3/envs/mlops mlops.conda-lock.yml \
    --conda /home/nelius/miniconda3/bin/conda

[Titus-von-Koeller]

Wow, that actually worked :star_struck: Thanks!

[mariusvniekerk]

@Titus-von-Koeller if you feel up to it i think an excellent PR would be for a FAQ style document for more complex edge cases like this. Perfectly fine if its the only case in that FAQ

[Titus-von-Koeller]

@Titus-von-Koeller if you feel up to it i think an excellent PR would be for a FAQ style document for more complex edge cases like this. Perfectly fine if its the only case in that FAQ

Yes, that's a great idea. I won't get around to it in the next three weeks, but then I'll think of something.