Longer Conda Build Times

[tran-david]

I've recently noticed that running conda env create -f environment.yml is taking longer to create the artic-ncov19 environment. Running it a few days ago took about 6hours, and then removing and recreating the environment a few days later took more than 15hrs.

Has anyone else experienced this issue or have any tips to speed up this process? Currently, the conda build process has been for 21hrs and is still going for me.

I'm using conda 4.10 and it appears to be stuck at the "solving environment" step. Running debug shows that the command doesn't hang, but is taking a while to search for available packages in the listed channels under environment.yml.

[leocaserta]

I have the same question. Every time I try to create the environment I get stuck at the "solving environment" step.

[rpetit3]

Might be worth giving Mamba (https://github.com/mamba-org/mamba) a try.

conda install -c conda-forge mamba
mamba create -f environment.yml

[nickloman]

I am having the same problem on Linux (OK on Mac). The Conda build process is a bit weird. I am wondering if pushing a new point release might help it along.

[vappiah]

I also have the same experience.

[rpetit3]

Give Mamba a try if you can. Using Mamba took ~1 minute to create the environment.

conda install -c conda-forge mamba
git clone git@github.com:artic-network/artic-ncov2019.git
time mamba env create -f artic-ncov2019/environment.yml

... mamba install text ...

#
# To activate this environment, use
#
#     $ conda activate artic-ncov2019
#
# To deactivate an active environment, use
#
#     $ conda deactivate

real    1m12.329s
user    0m50.265s
sys     0m10.918s

[vappiah]

I tried installing mamba and it is taking hours solve the environment. What is interesting is that I am able to install other bioinformatics packages.

[vappiah]

@rpetit3 Which anaconda version did you get the mamba installed on successfully?

[rpetit3]

I'm using MiniConda3, and here's the version.

conda --version
conda 4.10.1

base env

conda env export
name: base
channels:
  - conda-forge
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=4.5=1_gnu
  - brotlipy=0.7.0=py38h27cfd23_1003
  - bzip2=1.0.8=h7f98852_4
  - c-ares=1.17.1=h7f98852_1
  - ca-certificates=2021.5.25=h06a4308_1
  - certifi=2021.5.30=py38h06a4308_0
  - cffi=1.14.5=py38h261ae71_0
  - chardet=4.0.0=py38h06a4308_1003
  - conda=4.10.1=py38h06a4308_1
  - conda-package-handling=1.7.3=py38h27cfd23_1
  - cryptography=3.4.7=py38hd23ed53_0
  - icu=68.1=h58526e2_0
  - idna=2.10=pyhd3eb1b0_0
  - krb5=1.19.1=hcc1bbae_0
  - ld_impl_linux-64=2.35.1=h7274673_9
  - libarchive=3.5.1=hccf745f_2
  - libcurl=7.77.0=h2574ce0_0
  - libedit=3.1.20191231=he28a2e2_2
  - libev=4.33=h516909a_1
  - libffi=3.3=he6710b0_2
  - libgcc-ng=9.3.0=h5101ec6_17
  - libgomp=9.3.0=h5101ec6_17
  - libiconv=1.16=h516909a_0
  - libnghttp2=1.43.0=h812cca2_0
  - libsolv=0.7.18=h780b84a_0
  - libssh2=1.9.0=ha56f1ee_6
  - libstdcxx-ng=9.3.0=hd4cf53a_17
  - libxml2=2.9.12=h72842e0_0
  - lz4-c=1.9.3=h9c3ff4c_0
  - lzo=2.10=h516909a_1000
  - mamba=0.13.0=py38h2aa5da1_0
  - ncurses=6.2=he6710b0_1
  - openssl=1.1.1k=h27cfd23_0
  - pycosat=0.6.3=py38h7b6447c_1
  - pycparser=2.20=py_2
  - pyopenssl=20.0.1=pyhd3eb1b0_1
  - pysocks=1.7.1=py38h06a4308_0
  - python=3.8.5=h7579374_1
  - python_abi=3.8=1_cp38
  - readline=8.1=h27cfd23_0
  - reproc=14.2.1=h36c2ea0_0
  - reproc-cpp=14.2.1=h58526e2_0
  - requests=2.25.1=pyhd3eb1b0_0
  - ruamel_yaml=0.15.100=py38h27cfd23_0
  - setuptools=52.0.0=py38h06a4308_0
  - six=1.15.0=py38h06a4308_0
  - sqlite=3.35.4=hdfb4753_0
  - tk=8.6.10=hbc83047_0
  - tqdm=4.59.0=pyhd3eb1b0_1
  - urllib3=1.26.4=pyhd3eb1b0_0
  - xz=5.2.5=h7b6447c_0
  - yaml=0.2.5=h7b6447c_0
  - zlib=1.2.11=h7b6447c_3
  - zstd=1.5.0=ha95c52a_0
prefix: /home/robert_petit/miniconda3

[jvolkening]

Just want to say that Mamba was definitely the solution for me -- thanks @rpetit3. I now have high hopes that Mamba may solve a host of issues with Conda chewing up CI minutes (hours) while "solving environment" in Conda-based containers.

[FairerBadge66]

I was losing my mind trying to get the conda setup to work. Thank you very much @rpetit3 for your help.

[ankeetkumar]

I am sorry, I don't know if it is the right issue to ask this question. I am installing the artic pipeline for the first time. I was also having the same issue and the command that @rpetit3 gave did solveed the problem of creating environment.

But I want to ask, when I run guppy_basecaller command it doesn't run and shows this error "guppy_basecaller: command not found". I am under the notion that artic pipeline has guppy base caller installed as inbuilt function? is that true?