Exporting clean environment to environment.yml

[eakl]

Hi,

I have several questions about exporting a conda environment. When I'm inside my env, I use conda env export > environment.yml but:

1. How can I export my environment without version numbers and with only main packages (without their dependencies. As dependencies will be installed when packages are being installed). In other words, a 'clean' env like this:

name: my-env
dependencies:
 - python
 - ipython
- pandas
- scikit-learn
- pip:
    - jupyter_kernel_gateway

2. The last line is the prefix. What is it or what is its purpose? (I could not find the answer in the doc) and how can I prevent conda to export this line with my folder path in 'clear' as it it not 'sharing ready'.

If I missed something from the doc, please refer to it. Thanks

[olgabot]

The best thing that worked for me was piping it to cut to remove everything after an equals sign and then manually removing the last line:

$ conda env export | cut -f 1 -d '='
name: anchor-env
channels:
- bioconda
- r
- defaults
- conda-forge
dependencies:
- pycodestyle
- python-dateutil
- coverage
- cycler
- flake8
- freetype
- icu
- libpng
- matplotlib
- mccabe
- mkl
- numpy
- openssl
- pandas
- pip
- py
- pyflakes
- pyparsing
- pyqt
- pytest
- python
- pytz
- qt
- readline
- scikit-learn
- scipy
- seaborn
- setuptools
- sip
- six
- sqlite
- tk
- wheel
- xz
- zlib
- pip:
  - anchor-bio
  - python-coveralls
  - pyyaml
  - requests
prefix: /Users/olgabot/anaconda3/envs/anchor-env

Hope that helps.

[buchanae]

I have the same question. What's the point of an environment file with a hard-coded, site-specific prefix?

[minhncedutw]

Hi everyone,

I want to export environment.yml like this:

name: torch
channels:
  - defaults
  - conda-forge
dependencies:
  - numpy=1.15.2
  - numpy-base=1.15.2
  - pip=10.0.1
  - python=3.6.6
  - pip:
    - pillow=5.2.0
    - six=1.11.0
    - torch=0.4.1
    - torchvision=0.2.1

The normal export have 2 suffix, @olgabot 's method eliminate all 2. I want to keep just 1 suffix of version. Is it able?

[itsalitee]

Hi everyone,

I want to export environment.yml like this:

name: torch
channels:
  - defaults
  - conda-forge
dependencies:
  - numpy=1.15.2
  - numpy-base=1.15.2
  - pip=10.0.1
  - python=3.6.6
  - pip:
    - pillow=5.2.0
    - six=1.11.0
    - torch=0.4.1
    - torchvision=0.2.1

The normal export have 2 suffix, @olgabot 's method eliminate all 2. I want to keep just 1 suffix of version. Is it able?

Try this and see if it works for ya! Also, this includes automatic removal of the prefix @olgabot conda env export | cut -f -2 -d "=" | grep -v "prefix" > environment.yml

[avi-eko]

The normal export have 2 suffix, @olgabot 's method eliminate all 2. I want to keep just 1 suffix of version. Is it able?

@itsalitee -- Use the existing argument --no-builds to keep just the leading version numbers. The prefix line still needs to be removed:

conda env export --no-builds | grep -v "prefix" > environment.yml

[Cstyler]

with only main packages (without their dependencies. As dependencies will be installed when packages are being installed)

how to do this?

[padmano]

I have the same question too.

If there is going to be hard coding what is the purpose of it in environment file.

Does removing it in the file cause any issues down the line ?

Or is there any way to say %ANACONDA_HOME%/env/ in place of the URI.

[avi-eko]

There seems not to be a way (so far) to create a 'clean' or 'minimal' environment.yml automatically for conda environments, with just main packages and not dependencies listed. For some use cases, you might actually want all the dependencies explicitly, but probably not for public repos on github that you want to share.

There is a way to create a minimal requirements.txt file for pip. Try https://github.com/bndr/pipreqs. It doesn't work with Jupyter notebooks without using nbconvert to make *.py versions first.

So you could use conda for creating environments and pip for package management. It's not what I really want but it might work for some.

Another option is to make a maximal environment.yml as described above and then remove lines that don't have a corresponding line produced by pipreqs. I'm sure someone could do this nicely with a shell script.

[drozzy]

Update: June 2020 - I no longer think this is a good idea. I’ve been using Conda named environments and the explicit name has saved me many times from inadvertently screwing up the current environment!

I agree this is really wrong. In the official docs there is a section on "Sharing environments", yet it hard-codes the prefix and also the environment name: https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#sharing-an-environment

I don't see the reason for either of those. Here is how it should work (let the user specify the name and use default prefix from the system):

Env file like this (enviornment.yml) - notice NO name or prefix:

channels:
  - anaconda
  - conda-forge
  - defaults
dependencies:
  - altair=3.2.0=py37_0
  - python=3.7.3=h0371630_0
  - pip=19.3.1
  - pip:
    - matplotlib==3.1.1
    - tensorflow-datasets==1.3.0
    - tensorflow-gpu==2.0.0

should be created with this:

conda env create --name my_env -f environment.yml

similarly for update, no need to specify the name at all (just assume the user wishes to update the current environment):

conda env update --file environment.yml  --prune

[KingDarBoja]

The closest I could get to what the author wants is by using the --from-history flag along with the cutting prefix approach stated above:

conda env export --from-history | grep -v "prefix" > environment.yml

Generates:

name: envname
channels:
  - defaults
dependencies:
  - python=3.7
  - numpy
  - nomkl
  - flask
  - flask-cors
  - pandas
  - psycopg2
  - tqdm
  - pyjwt
  - graphene
  - flask-graphql
  - pywin32
  - jupyter
  - typing
  - xlrd

Documentation explains what --from-history flag does.

[robertlugg]

There are so many variations in use model. @KingDarBoja may be closest to answer the OP's question. In many cases, I want a user to use exactly my environment. This way we have the same setup. If they notice a behavior, I will be able to repeat that same behavior in my environment. This removes one class of inconsistencies.

It does seem that we still can't get a basic list of things we might want. For instance for KingDarBoja's example: From which channel did each package come from and which version was installed? When there is only one channel, no problem, but otherwise how do I know where it came from?

I suggest that we need a proposal of how an export command would look like which meets all and makes the most common uses easy.

[KingDarBoja]

The starting point is located at conda_env/cli/main_export.py file, which handles the entire cli options for the export command.

These options are passed to from_environment function, whose definition is located at conda_env/env.py file.

[gwerbin]

Hi everyone,

I want to export environment.yml like this:

name: torch
channels:
  - defaults
  - conda-forge
dependencies:
  - numpy=1.15.2
  - numpy-base=1.15.2
  - pip=10.0.1
  - python=3.6.6
  - pip:
    - pillow=5.2.0
    - six=1.11.0
    - torch=0.4.1
    - torchvision=0.2.1

The normal export have 2 suffix, @olgabot 's method eliminate all 2. I want to keep just 1 suffix of version. Is it able?

This should be the default output format IMO. Maybe with name removed. Is there any reason it should be otherwise?

At least if Conda exposed this functionality in some public API then people could write their own tools.

[fjhheras]

The closest I could get to what the author wants is by using the --from-history flag along with the cutting prefix approach stated above:

conda env export --from-history | grep -v "prefix" > environment.yml

This is great! I realised it does not export packages installed with pip, though.

[kadirmalak]

The best thing that worked for me was piping it to cut to remove everything after an equals sign and then manually removing the last line:

$ conda env export | cut -f 1 -d '='
name: anchor-env
channels:
- bioconda
- r
- defaults
- conda-forge
dependencies:
- pycodestyle
- python-dateutil
- coverage
- cycler
- flake8
- freetype
- icu
- libpng
- matplotlib
- mccabe
- mkl
- numpy
- openssl
- pandas
- pip
- py
- pyflakes
- pyparsing
- pyqt
- pytest
- python
- pytz
- qt
- readline
- scikit-learn
- scipy
- seaborn
- setuptools
- sip
- six
- sqlite
- tk
- wheel
- xz
- zlib
- pip:
  - anchor-bio
  - python-coveralls
  - pyyaml
  - requests
prefix: /Users/olgabot/anaconda3/envs/anchor-env

Hope that helps.

Very clean job :) By the way you can remove the last line with another piping...

| grep -v ^prefix

[itsergiu]

Building identical conda environments https://docs.conda.io/projects/conda/en/4.6.0/user-guide/tasks/manage-environments.html#building-identical-conda-environments conda list --explicit > spec-file.txt

[ashleymaeconard]

The best thing that worked for me was piping it to cut to remove everything after an equals sign and then manually removing the last line:

$ conda env export | cut -f 1 -d '='
name: anchor-env
channels:
- bioconda
- r
- defaults
- conda-forge
dependencies:
- pycodestyle
- python-dateutil
- coverage
- cycler
- flake8
- freetype
- icu
- libpng
- matplotlib
- mccabe
- mkl
- numpy
- openssl
- pandas
- pip
- py
- pyflakes
- pyparsing
- pyqt
- pytest
- python
- pytz
- qt
- readline
- scikit-learn
- scipy
- seaborn
- setuptools
- sip
- six
- sqlite
- tk
- wheel
- xz
- zlib
- pip:
  - anchor-bio
  - python-coveralls
  - pyyaml
  - requests
prefix: /Users/olgabot/anaconda3/envs/anchor-env

Hope that helps.

Very clean job :) By the way you can remove the last line with another piping...

| grep -v ^prefix

These are great. I would also add that if you want to keep the versions you simply change -f 1 to -f-2. All together that is: conda env export | cut -f-2 -d '='|grep -v ^prefix.

[ATelders]

The closest I could get to what the author wants is by using the --from-history flag along with the cutting prefix approach stated above:

conda env export --from-history | grep -v "prefix" > environment.yml

Thank you for this!

On Windows, you just need to replace grep with findstr

conda env export --from-history | findstr -v "prefix" > environment.yml

For me this command also exported the pip packages.

[github-actions[bot]]

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

1. Verify that you can still reproduce the issue at hand
2. Comment that the issue is still reproducible and include:
   • What OS and version you reproduced the issue on
   • What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!

[gwerbin]

This is still an open feature request!

[krikru]

While it doesn't solve the problem completely, I find the following command for printing all manually installed packages helpful:

cat $CONDA_PREFIX/conda-meta/history | grep " install "

For me, this results in

# cmd: /home/krikru/installed_software/mambaforge/bin/mamba install python
# cmd: /home/krikru/installed_software/mambaforge/bin/mamba install pytorch torchvision torchaudio cudatoolkit=11.6 -c pytorch -c conda-forge
# cmd: /home/krikru/installed_software/mambaforge/bin/mamba install matplotlib
# cmd: /home/krikru/installed_software/mambaforge/bin/mamba install iris
# cmd: /home/krikru/installed_software/mambaforge/bin/mamba install ncview
# cmd: /home/krikru/installed_software/mambaforge/bin/mamba install ipython
# cmd: /home/krikru/installed_software/mambaforge/bin/mamba install tqdm
# cmd: /home/krikru/installed_software/mambaforge/bin/mamba install xarray
# cmd: /home/krikru/installed_software/mambaforge/bin/mamba install appdirs

which are all commands that I have executed (except for the part up to "mamba").