Conda environment file is unsolvable

[mwhamgenomics]

When I try to run the conda env create -f hyb2.yml setup step, Conda runs for about 30mins-1hr trying to solve the environment before exiting with an error:

Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working...
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

Solving environment: ...working...
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Package make conflicts for:
bioconductor-biocparallel=1.28.3 -> r-base[version='>=4.1,<4.2.0a0'] -> make
r-pillar=1.7.0 -> r-base[version='>=4.1,<4.2.0a0'] -> make
r-fastmap=1.1.0 -> r-base[version='>=4.1,<4.2.0a0'] -> make
[...]

Then a lot of package conflict messages:

Package lerc conflicts for:
lerc=3.0
libtiff=4.3.0 -> lerc[version='>=2.2.1,<3.0a0|>=3.0,<4.0a0'] 
r-base=4.1.3 -> libtiff[version='>=4.6.0,<4.7.0a0'] -> lerc[version='>=2.2.1,<3.0a0|>=3.0,<4.0a0|>=4.0.0,<5.0a0']
The following specifications were found to be incompatible with your system:

  - feature:/linux-64::__glibc==2.17=0
  - feature:|@/linux-64::__glibc==2.17=0
  - bioconductor-biobase=2.54.0 -> libgcc-ng[version='>=10.3.0'] -> __glibc[version='>=2.17']
  - bioconductor-biocparallel=1.28.3 -> libgcc-ng[version='>=10.3.0'] -> __glibc[version='>=2.17']
[...]

I think there are two problems:

1. There are too many package specifications in bin/hyb2.yml. Most of them look like sub-dependencies and can be removed. Only the main ones that you would manually run conda install for need to be specified
2. bin/hyb2.yml specifies exact builds, which may conflict with the requirements of other compute systems. The build IDs can be removed, e.g. cairo=1.16.0=ha12eb4b_1010 only needs to be cairo=1.16.0

[mwhamgenomics]

From looking at the scripts and trying to run this pipeline, the only dependencies required are:

unafold

I already had it on my system, but other people might not - see #4

faToTwoBit

wget http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/faToTwoBit
chmod u+x faToTwoBit

blast

wget https://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ncbi-blast-2.14.1+-x64-linux.tar.gz
tar -zxf ncbi-blast-2.14.1+-x64-linux.tar.gz

bowtie2

wget https://github.com/BenLangmead/bowtie2/releases/download/v2.3.4.3/bowtie2-2.3.4.3-linux-x86_64.zip
unzip bowtie2-2.3.4.3-linux-x86_64.zip

R + dependencies

conda create -p path/to/R-conda -c conda-forge r
conda activate path-to-R-conda
conda install -c conda-forge r-data.table
conda install -c conda-forge r-ggplot2
conda install -c conda-forge r-gridextra
conda install -c conda-forge -c bioconda bioconductor-deseq2

Perl + dependencies

Bio::SeqIO - only needed in pf.pl, which is never called - is this required?

Python + dependencies

pandas - used in DESeq_interaction_split_select.py, but none of the scripts that run this ever get called - is this required?

For some reason, Conda couldn't solve the environment when running conda create r-ggplot2 r-gridextra ..., but it worked fine installing things one at a time - will keep investigating that.

[Jylau14]

I will upload unafold onto here as well.

Do you think it's better to load blast and bowtie2 onto conda as well?

I've deleted pf.pl.

DESeq_interaction_split_select.py is only used in hyb2_compare.

[Jylau14]

Added your blast and bowtie2 installation commands to the homepage.

Removed specific builds for hyb2.yml.

Please check if it is working.

Thank you!