jonashaag
commented
1 year ago We will be adding that soonish, maybe in the next couple of weeks/months. If you need it more urgently I’m happy to help you to come up with a patch :)
Open Yura52 opened 1 year ago
jonashaag
commented
1 year ago We will be adding that soonish, maybe in the next couple of weeks/months. If you need it more urgently I’m happy to help you to come up with a patch :)
jonashaag
commented
1 year ago Also do you know conda-lock?
Yura52
commented
1 year ago Thank you for the quick reply! I was not aware of conda-lock, though after a quick look my impression is that conda-lock lacks this feature.
Also, it turned out that my case might be a bit more complicated than I thought, and now, I am less sure that the proposed idea is the one that would solve my issue.
jonashaag
commented
1 year ago OK! Feel free to let us know here what's your use case.
Btw, moving to different architectures may be complicated because packages generally have other dependencies on other architectures. Eg. some architectures might have MKL and others only have OpenBLAS.
Yura52
commented
1 year ago Before I start, I would like to say that micromamba is such a cool thing! It is such a relief to have a single fast binary for managing environments.
Also, it looks like the above discussion about architectures does not apply to my case, sorry for the misleading first post. Both machines I have fall into the linux-64 category.
My setup is specific and makes it difficuilt to provide consice minimal examples, so I will talk only about one specific issue that to some extent illustrates the spirit of the problems.
On machine A, I installed PyTorch v1.13.1 with CUDA support with this command: micromamba install pytorch pytorch-cuda=11.7 -c pytorch -c nvidia -с conda-forge. Then I installed many more packages with a couple of micromamba install commands.
One of my attempts to reproduce the environment on machine B was based on this environment.yaml file:
Below, I provide the result of micromamba install -f environment.yaml. The problem is that the suggested PyTorch is CPU-only and comes from the conda-forge channel.
So instead of environment.yaml file, I currently maintain a sequence of micromamba install commands with the right channels and package versions. Overall, this workaround is good enough for me.
P.S. I have just encountered this issue. So there is a syntax for package-wise channels or is it a conda-specific thing?
jonashaag
commented
1 year ago Quick reply to your question, there is channel::package but I’m not sure it works currently
jonashaag
commented
1 year ago Also I think you should be able to force GPU using pytorch =… =*cuda*
Yura52
commented
1 year ago Wow, I was not aware of this syntax. Is there a documentation for theenvironment.yaml format?
jonashaag
commented
1 year ago I don’t find the Conda docs particularly approachable but here is some documentation on the version spec part https://docs.conda.io/projects/conda-build/en/stable/resources/package-spec.html#package-match-specifications
Yura52
commented
1 year ago @jonashaag hi! The --channel-subdir is a great addition to micromamba 1.5.1! Do I understand correctly that this does not change the state of this specific issue? I mean this part:
I would like to export an environment and then reproduce it on a different architecture. Currently, it looks impossible
Because the result of -subdir part looks platform-specific.
jonashaag
commented
1 year ago You can't because some packages are not available on some architectures.
Can you please provide info on your exact use case?
Yura52
commented
1 year ago First, I should say that my issue is not urgent nor critical, so if it feels like out of scope, feel free to close this issue. In any case, the new --channel-subdir is already a great feature that will improve my setup.
I am a machine learning researcher, and it often happens that I work on one project using two machines:
I wonder what is the best way to maintain environment files in this case to keep the two machines somewhat in sync? Additional context:
My current solution (simplified for the sake of clarity) is to manually compose and support environment-macos.yaml file where I only list packages directly imported by my code.
jonashaag
commented
1 year ago Having some packages installed for some platforms only is not supported by Conda syntax. Although it is supported by a Mamba specific feature: sel(...). I think you can find an example in one of the environment files in this very repo.
I think the best you can do is a relatively strictly pinned environment file.
Yura52
commented
1 year ago I see, thanks for the prompt reply!
Hi! Thanks for the great project!
I personally use micromamba, but the question applies to everything: conda, mamba, micromamba.
I would like to export an environment and then reproduce it on a different architecture. Currently, it looks impossible:
micromamba env exportincludes build information and produces architecture-specific filesmicromamba env export --explicitmicromamba env export --no-builddoes not save from what channel each package comes from (the "channels" section is not enough in that sense)The
exportcommand is not crucial here, I am ready to assemble the file by hand if the format allows including channel information.So basically I am looking for something like
micromamba env export --save-channel-for-each-package