search

maxplanck-ie / snakepipes

Customizable workflows based on snakemake and python for the analysis of NGS data
http://snakepipes.readthedocs.io
378 stars 85 forks source link

createIndices fails #493

Closed adeslatt closed 4 years ago

adeslatt commented 4 years ago

Hello.

On a mac OS 10. I am stymied by the createIndices step

(snakePipes) x86_64-apple-darwin13% createIndices -v --local True --tempdir ~/Desktop/code/output-dir -o output-dir --genome ftp://ftp.ensembl.org/pub/release-97/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz --gtf ftp://ftp.ensembl.org/pub/release-97/gtf/homo_sapiens/Homo_sapiens.GRCh38.97.gtf.gz  GRCh38_release97

Error I get:

(snakePipes) x86_64-apple-darwin13% createIndices -v --local --tools bwameth --tempdir ~/Desktop/code/output-dir -o output-dir --genome ftp://ftp.ensembl.org/pub/release-97/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz --gtf ftp://ftp.ensembl.org/pub/release-97/gtf/homo_sapiens/Homo_sapiens.GRCh38.97.gtf.gz  GRCh38_release97

Failed to create temp dir under temp path prefix (~/Desktop/code/output-dir)! Try fallback: ~/Desktop/code/output-dir/ ...

Also failed to create temp dir under fallback prefix (~/Desktop/code/output-dir/)!
(snakePipes) x86_64-apple-darwin13% createIndices -v --local True --tempdir ~/Desktop/code/output-dir -o output-dir --genome ftp://ftp.ensembl.org/pub/release-97/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz --gtf ftp://ftp.ensembl.org/pub/release-97/gtf/homo_sapiens/Homo_sapiens.GRCh38.97.gt

All previous steps in install were successful -- with a minor difference in that on a mac you need to activate environments a little differently -- it could be in the end my mac is too underpowered to run this and I will need to run this on a cluster -- but am trying to see if there are somethings that might be accomplished on the mac as well....

Here are my steps:

  1. installed miniconda3 in ~/Desktop/conda3 

    wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh -O ~/miniconda.sh
bash ~/miniconda.sh -b -p ~/Desktop/conda3

  2. Created a python3 environment with

    ~/Desktop/conda3/bin/conda create -n mypython3 python=3

  3. Then I was able to install snakePipes - the “-n” names the environment — the ‘-c’ uses channels

    conda create -n snakePipes -c mpi-ie -c bioconda -c conda-forge snakePipes

On the mac — to activate an environment you need to source that environment

source ~/Desktop/conda3/bin/activate snakePipes
  1. Then the installation instructions state that one needs to create the environments for each of the pipelines: 
    snakePipes createEnvs

This failed for me. Giving me the error about not finding r-histonehmm pkg:

(snakePipes) > snakePipes createEnvs
Creating environment from ~/Desktop/conda3/envs/snakePipes/lib/python3.6/site-packages/snakePipes/shared/rules/envs/shared.yaml in ~/Desktop/conda3/envs
Creating environment from ~/Desktop/conda3/envs/snakePipes/lib/python3.6/site-packages/snakePipes/shared/rules/envs/createIndices.yaml in ~/Desktop/conda3/envs
Creating environment from ~/Desktop/conda3/envs/snakePipes/lib/python3.6/site-packages/snakePipes/shared/rules/envs/rna_seq.yaml in ~/Desktop/conda3/envs
Creating environment from ~/Desktop/conda3/envs/snakePipes/lib/python3.6/site-packages/snakePipes/shared/rules/envs/sc_rna_seq.yaml in ~/Desktop/conda3/envs
Creating environment from ~/Desktop/conda3/envs/snakePipes/lib/python3.6/site-packages/snakePipes/shared/rules/envs/dna_mapping.yaml in ~/Desktop/conda3/envs
Creating environment from ~/Desktop/conda3/envs/snakePipes/lib/python3.6/site-packages/snakePipes/shared/rules/envs/chip_seq.yaml in ~/Desktop/conda3/envs
Creating environment from ~/Desktop/conda3/envs/snakePipes/lib/python3.6/site-packages/snakePipes/shared/rules/envs/histone_hmm.yaml in ~/Desktop/conda3/envs
Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound: 
  - r-histonehmm=1.7-1

This is how that was fixed: conda install -c bioconda -c conda-forge r-histonehmm

I had to edit the yaml file for the required package because it actually specified a version – this is a bit of a hack – and should be fixed properly – I’ll message the author and let him know. I use emacs – but the file was this one:

emacs ~/conda3/envs/snakePipes/lib/python3.6/site-packages/snakePipes/shared/rules/envs/histone_hmm.yaml

Its contents had been:

name: snakepipes_histoneHMM_1.0
channels:
 - conda-forge
 - bioconda
dependencies:
 - r-histonehmm = 1.7-1

I changed it to:

name: snakepipes_histoneHMM_1.0
channels:
 - conda-forge
 - bioconda
dependencies:
 - r-histonehmm

This should have allowed me to run the createIndices command but it fails. Any hints or help would be appreciated! Thanks!

dpryan79 commented 4 years ago

Please try creating that failed environment again in ~20 minutes. I've pushed out a new build of r-histonehmm on OSX over on bioconda, so it should work now. Thanks for pointing out the issue with 1.7-1 as a version, it should be 1.7.1 actually, though apparently conda knows what we meant since it's able to solve the environment on Linux.

dpryan79 commented 4 years ago

We're now testing that this works with every commit, so I hope you never run into such problems again :)