search

maxplanck-ie / snakepipes

Customizable workflows based on snakemake and python for the analysis of NGS data
http://snakepipes.readthedocs.io
381 stars 85 forks source link

problems in createIndicies #804

Closed Leif78 closed 2 years ago

Leif78 commented 2 years ago

Hi all, I try to run createIndicies, but it fails. Seems like an error in conda environment?

Gratefull for help!

log: (snakePipes) leif@leif-OptiPlex-9020:~/Desktop$ createIndices -o ~/Documents --genomeURL ~/Documents/GRCz11/Danio_rerio.GRCz11.dna_sm.primary_assembly.fa --gtfURL ~/Documents/GRCz11/Danio.gtf --local GRCz11

---- This analysis has been done using snakePipes version 2.5.3 ---- Building DAG of jobs... Using shell: /usr/bin/bash Provided cores: 5 Rules claiming more threads will be scaled down. Job counts: count jobs 1 all 1 bowtie2Index 1 bwaIndex 1 bwamethIndex 1 computeEffectiveGenomeSize 1 extendGenicRegions 1 fastaDict 1 fastaIndex 1 gtf2BED 1 hisat2Index 1 make2bit 1 makeKnownSpliceSites 1 starIndex 13

[Mon Dec 13 11:46:26 2021] rule computeEffectiveGenomeSize: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/genome_fasta/effectiveSize log: logs/computeEffectiveGenomeSize.log jobid: 14

[Mon Dec 13 11:46:26 2021] rule fastaDict: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/genome_fasta/genome.dict log: logs/fastaDict.log jobid: 3

[Mon Dec 13 11:46:26 2021] rule bwamethIndex: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/BWAmethIndex/genome.fa.bwameth.c2t.sa log: logs/bwamethIndex.log jobid: 13

[Mon Dec 13 11:46:26 2021] rule fastaIndex: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/genome_fasta/genome.fa.fai log: logs/fastaIndex.log jobid: 2

[Mon Dec 13 11:46:26 2021] rule make2bit: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/genome_fasta/genome.2bit log: logs/make2bit.log jobid: 4

Activating conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Activating conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Activating conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 /usr/bin/bash: seqtk: command not found [Mon Dec 13 11:46:32 2021] [Mon Dec 13 11:46:32 2021] Error in rule make2bit: Error in rule computeEffectiveGenomeSize: jobid: 4 jobid: 14 output: /home/leif/Documents/genome_fasta/genome.2bit output: /home/leif/Documents/genome_fasta/effectiveSize log: logs/make2bit.log (check log file(s) for error message) log: logs/computeEffectiveGenomeSize.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 conda-env: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 shell:

    faToTwoBit /home/leif/Documents/genome_fasta/genome.fa /home/leif/Documents/genome_fasta/genome.2bit 2> logs/make2bit.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
shell:

    seqtk comp /home/leif/Documents/genome_fasta/genome.fa | awk '{tot += $3 + $4 + $5 + $6}END{print tot}' > /home/leif/Documents/genome_fasta/effectiveSize 2> logs/computeEffectiveGenomeSize.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs. Removing output files of failed job computeEffectiveGenomeSize since they might be corrupted: /home/leif/Documents/genome_fasta/effectiveSize Job failed, going on with independent jobs.

[Mon Dec 13 11:46:32 2021] rule gtf2BED: input: /home/leif/Documents/annotation/genes.gtf output: /home/leif/Documents/annotation/genes.bed log: logs/gtf2BED.log jobid: 6

[Mon Dec 13 11:46:32 2021] rule bwaIndex: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/BWAIndex/genome.fa.sa log: logs/bwaIndex.log jobid: 12

Not a conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Not a conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 [Mon Dec 13 11:46:32 2021] Error in rule fastaDict: jobid: 3 output: /home/leif/Documents/genome_fasta/genome.dict log: logs/fastaDict.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 shell:

    samtools dict -o /home/leif/Documents/genome_fasta/genome.dict /home/leif/Documents/genome_fasta/genome.fa 2> logs/fastaDict.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs. [Mon Dec 13 11:46:32 2021] Error in rule fastaIndex: jobid: 2 output: /home/leif/Documents/genome_fasta/genome.fa.fai log: logs/fastaIndex.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 shell:

    samtools faidx /home/leif/Documents/genome_fasta/genome.fa 2> logs/fastaIndex.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs. Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 ln: failed to create symbolic link '/home/leif/Documents/BWAmethIndex/genome.fa': File exists [Mon Dec 13 11:46:33 2021] Error in rule bwamethIndex: jobid: 13 output: /home/leif/Documents/BWAmethIndex/genome.fa.bwameth.c2t.sa log: logs/bwamethIndex.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    ln -s /home/leif/Documents/genome_fasta/genome.fa /home/leif/Documents/BWAmethIndex/genome.fa
    bwameth.py index /home/leif/Documents/BWAmethIndex/genome.fa 2> logs/bwamethIndex.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs. Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 /usr/bin/bash: gtfToGenePred: command not found ln: failed to create symbolic link '/home/leif/Documents/BWAIndex/genome.fa': File exists [Mon Dec 13 11:46:34 2021] Error in rule bwaIndex: jobid: 12 output: /home/leif/Documents/BWAIndex/genome.fa.sa log: logs/bwaIndex.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    ln -s /home/leif/Documents/genome_fasta/genome.fa /home/leif/Documents/BWAIndex/genome.fa
    bwa index /home/leif/Documents/BWAIndex/genome.fa 2> logs/bwaIndex.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs. [Mon Dec 13 11:46:34 2021] Error in rule gtf2BED: jobid: 6 output: /home/leif/Documents/annotation/genes.bed log: logs/gtf2BED.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    awk '{if ($3 != "gene") print $0;}' /home/leif/Documents/annotation/genes.gtf             | grep -v "^#"             | gtfToGenePred /dev/stdin /dev/stdout             | genePredToBed stdin /home/leif/Documents/annotation/genes.bed 2> logs/gtf2BED.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Mon Dec 13 11:46:34 2021] rule hisat2Index: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/HISAT2Index/genome.6.ht2 log: logs/hisat2Index.log jobid: 9 threads: 5

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 ln: failed to create symbolic link '/home/leif/Documents/HISAT2Index/genome.fa': File exists [Mon Dec 13 11:46:36 2021] Error in rule hisat2Index: jobid: 9 output: /home/leif/Documents/HISAT2Index/genome.6.ht2 log: logs/hisat2Index.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    ln -s /home/leif/Documents/genome_fasta/genome.fa /home/leif/Documents/HISAT2Index/genome.fa
    hisat2-build -q -p 5 /home/leif/Documents/HISAT2Index/genome.fa /home/leif/Documents/HISAT2Index/genome
    2> logs/hisat2Index.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Mon Dec 13 11:46:36 2021] rule bowtie2Index: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/BowtieIndex/genome.rev.2.bt2 log: logs/bowtie2Index.log jobid: 8 threads: 5

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 ln: failed to create symbolic link '/home/leif/Documents/BowtieIndex/genome.fa': File exists [Mon Dec 13 11:46:39 2021] Error in rule bowtie2Index: jobid: 8 output: /home/leif/Documents/BowtieIndex/genome.rev.2.bt2 log: logs/bowtie2Index.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    ln -s /home/leif/Documents/genome_fasta/genome.fa /home/leif/Documents/BowtieIndex/genome.fa
    bowtie2-build -t 5 /home/leif/Documents/BowtieIndex/genome.fa /home/leif/Documents/BowtieIndex/genome
    if [[ -f BowtieIndex/genome.rev.2.bt2l ]]; then ln -s genome.rev.2.bt2l /home/leif/Documents/BowtieIndex/genome.rev.2.bt2 ; fi
    2> logs/bowtie2Index.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Mon Dec 13 11:46:39 2021] rule starIndex: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/STARIndex/SAindex log: logs/starIndex.log jobid: 11 threads: 5

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 [Mon Dec 13 11:46:41 2021] Error in rule starIndex: jobid: 11 output: /home/leif/Documents/STARIndex/SAindex log: logs/starIndex.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    STAR --runThreadN 5 --runMode genomeGenerate --genomeDir /home/leif/Documents/STARIndex --genomeFastaFiles /home/leif/Documents/genome_fasta/genome.fa 2> logs/starIndex.log
    if [[ -w Log.out ]]; then rm -v Log.out; elif [[ -w /home/leif/Documents/STARIndex/Log.out ]]; then rm -v /home/leif/Documents/STARIndex/Log.out; fi

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Mon Dec 13 11:46:41 2021] rule makeKnownSpliceSites: input: /home/leif/Documents/annotation/genes.gtf output: /home/leif/Documents/HISAT2Index/splice_sites.txt log: logs/makeKnownSpliceSites.log jobid: 10 threads: 5

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 [Mon Dec 13 11:46:43 2021] Error in rule makeKnownSpliceSites: jobid: 10 output: /home/leif/Documents/HISAT2Index/splice_sites.txt log: logs/makeKnownSpliceSites.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    hisat2_extract_splice_sites.py /home/leif/Documents/annotation/genes.gtf > /home/leif/Documents/HISAT2Index/splice_sites.txt 2> logs/makeKnownSpliceSites.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job makeKnownSpliceSites since they might be corrupted: /home/leif/Documents/HISAT2Index/splice_sites.txt Job failed, going on with independent jobs. Exiting because a job execution failed. Look above for error message Complete log: /home/leif/Documents/.snakemake/log/2021-12-13T114626.327685.snakemake.log

!!! ERROR in index creation workflow! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Error: snakemake returned an error code of 1, so processing is incomplete!

adRn-s commented 2 years ago

I was unable to reproduce this error using Ensembl's GRCz11 fasta and gtf files. For me, the createIndices command worked flawlessly and completed it tasks after 2h17m. You should try reinstalling snakePipes since there are error messages of 'command not found' on the log you pasted.

LeilyR commented 2 years ago

please always send us the version and the commnad line you used as well the log of the failed rules. However from what you have sent I can alreayd see where the error comes from:

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5
Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5

A fresh installation on a fresh env, not on your base env, may already help you.

Leif78 commented 2 years ago

Thanks for answers. I now tried again. I deleted miniconda3, reinstalled miniconda3 and reinstalled snakePipes 5.2.3. Set PATH with export PATH="/path/to/miniconda3/bin:$PATH" CreateIndicies worked without problems.

my command: createIndices -o ~/Documents --genomeURL ~/Documents/GRCz11/Danio_rerio.GRCz11.dna_sm.primary_assembly.fa --gtfURL ~/Documents/GRCz11/Danio.gtf --local GRCz11

and the first error message: Activating conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Activating conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Activating conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Not a conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 [Wed Dec 15 12:20:54 2021] Error in rule fastaDict: jobid: 3 output: /home/leif/Documents/genome_fasta/genome.dict log: logs/fastaDict.log (check log file(s) for error message) Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5

It is the same error as before. Am I doing anything wrong in the set up?

adRn-s commented 2 years ago

You should re-run createEnvs as part of the snakePipes re-installation before running createIndices.

Leif78 commented 2 years ago

I re-installed miniconda and snakePipes and re-run createEnvs. No errors so far.

command: createIndices -o ~/Documents --genomeURL ~/Documents/GRCz11/Danio_rerio.GRCz11.dna_sm.primary_assembly.fa --gtfURL ~/Documents/GRCz11/Danio.gtf --local GRCz11

this produce the following error:

---- This analysis has been done using snakePipes version 2.5.3 ---- Building DAG of jobs... Using shell: /usr/bin/bash Provided cores: 5 Rules claiming more threads will be scaled down. Job counts: count jobs 1 all 1 bowtie2Index 1 bwaIndex 1 bwamethIndex 1 computeEffectiveGenomeSize 1 extendGenicRegions 1 fastaDict 1 fastaIndex 1 gtf2BED 1 hisat2Index 1 make2bit 1 makeKnownSpliceSites 1 starIndex 13

[Wed Dec 15 20:55:00 2021] rule bwaIndex: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/BWAIndex/genome.fa.sa log: logs/bwaIndex.log jobid: 12

[Wed Dec 15 20:55:00 2021] rule computeEffectiveGenomeSize: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/genome_fasta/effectiveSize log: logs/computeEffectiveGenomeSize.log jobid: 14

[Wed Dec 15 20:55:00 2021] rule fastaDict: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/genome_fasta/genome.dict log: logs/fastaDict.log jobid: 3

[Wed Dec 15 20:55:00 2021] rule make2bit: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/genome_fasta/genome.2bit log: logs/make2bit.log jobid: 4

[Wed Dec 15 20:55:00 2021] rule bwamethIndex: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/BWAmethIndex/genome.fa.bwameth.c2t.sa log: logs/bwamethIndex.log jobid: 13

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Activating conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 [Wed Dec 15 20:55:01 2021] Error in rule make2bit: jobid: 4 output: /home/leif/Documents/genome_fasta/genome.2bit log: logs/make2bit.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    faToTwoBit /home/leif/Documents/genome_fasta/genome.fa /home/leif/Documents/genome_fasta/genome.2bit 2> logs/make2bit.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Wed Dec 15 20:55:01 2021] rule fastaIndex: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/genome_fasta/genome.fa.fai log: logs/fastaIndex.log jobid: 2

Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 ln: failed to create symbolic link '/home/leif/Documents/BWAIndex/genome.fa': File exists [Wed Dec 15 20:55:01 2021] Error in rule bwaIndex: jobid: 12 output: /home/leif/Documents/BWAIndex/genome.fa.sa log: logs/bwaIndex.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    ln -s /home/leif/Documents/genome_fasta/genome.fa /home/leif/Documents/BWAIndex/genome.fa
    bwa index /home/leif/Documents/BWAIndex/genome.fa 2> logs/bwaIndex.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Wed Dec 15 20:55:01 2021] rule gtf2BED: input: /home/leif/Documents/annotation/genes.gtf output: /home/leif/Documents/annotation/genes.bed log: logs/gtf2BED.log jobid: 6

Activating conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 ln: failed to create symbolic link '/home/leif/Documents/BWAmethIndex/genome.fa': File exists [Wed Dec 15 20:55:01 2021] Error in rule bwamethIndex: jobid: 13 output: /home/leif/Documents/BWAmethIndex/genome.fa.bwameth.c2t.sa log: logs/bwamethIndex.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    ln -s /home/leif/Documents/genome_fasta/genome.fa /home/leif/Documents/BWAmethIndex/genome.fa
    bwameth.py index /home/leif/Documents/BWAmethIndex/genome.fa 2> logs/bwamethIndex.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs. Not a conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 /usr/bin/bash: seqtk: command not found [Wed Dec 15 20:55:01 2021] Error in rule computeEffectiveGenomeSize: jobid: 14 output: /home/leif/Documents/genome_fasta/effectiveSize log: logs/computeEffectiveGenomeSize.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 shell:

    seqtk comp /home/leif/Documents/genome_fasta/genome.fa | awk '{tot += $3 + $4 + $5 + $6}END{print tot}' > /home/leif/Documents/genome_fasta/effectiveSize 2> logs/computeEffectiveGenomeSize.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job computeEffectiveGenomeSize since they might be corrupted: /home/leif/Documents/genome_fasta/effectiveSize Job failed, going on with independent jobs. Not a conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 [Wed Dec 15 20:55:01 2021] Error in rule fastaDict: jobid: 3 output: /home/leif/Documents/genome_fasta/genome.dict log: logs/fastaDict.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 shell:

    samtools dict -o /home/leif/Documents/genome_fasta/genome.dict /home/leif/Documents/genome_fasta/genome.fa 2> logs/fastaDict.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs. Activating conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 [Wed Dec 15 20:55:02 2021] Error in rule fastaIndex: jobid: 2 output: /home/leif/Documents/genome_fasta/genome.fa.fai log: logs/fastaIndex.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 shell:

    samtools faidx /home/leif/Documents/genome_fasta/genome.fa 2> logs/fastaIndex.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs. Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 /usr/bin/bash: gtfToGenePred: command not found [Wed Dec 15 20:55:02 2021] Error in rule gtf2BED: jobid: 6 output: /home/leif/Documents/annotation/genes.bed log: logs/gtf2BED.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    awk '{if ($3 != "gene") print $0;}' /home/leif/Documents/annotation/genes.gtf             | grep -v "^#"             | gtfToGenePred /dev/stdin /dev/stdout             | genePredToBed stdin /home/leif/Documents/annotation/genes.bed 2> logs/gtf2BED.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Wed Dec 15 20:55:02 2021] rule bowtie2Index: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/BowtieIndex/genome.rev.2.bt2 log: logs/bowtie2Index.log jobid: 8 threads: 5

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 ln: failed to create symbolic link '/home/leif/Documents/BowtieIndex/genome.fa': File exists [Wed Dec 15 20:55:02 2021] Error in rule bowtie2Index: jobid: 8 output: /home/leif/Documents/BowtieIndex/genome.rev.2.bt2 log: logs/bowtie2Index.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    ln -s /home/leif/Documents/genome_fasta/genome.fa /home/leif/Documents/BowtieIndex/genome.fa
    bowtie2-build -t 5 /home/leif/Documents/BowtieIndex/genome.fa /home/leif/Documents/BowtieIndex/genome
    if [[ -f BowtieIndex/genome.rev.2.bt2l ]]; then ln -s genome.rev.2.bt2l /home/leif/Documents/BowtieIndex/genome.rev.2.bt2 ; fi
    2> logs/bowtie2Index.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Wed Dec 15 20:55:02 2021] rule starIndex: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/STARIndex/SAindex log: logs/starIndex.log jobid: 11 threads: 5

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 [Wed Dec 15 20:55:03 2021] Error in rule starIndex: jobid: 11 output: /home/leif/Documents/STARIndex/SAindex log: logs/starIndex.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    STAR --runThreadN 5 --runMode genomeGenerate --genomeDir /home/leif/Documents/STARIndex --genomeFastaFiles /home/leif/Documents/genome_fasta/genome.fa 2> logs/starIndex.log
    if [[ -w Log.out ]]; then rm -v Log.out; elif [[ -w /home/leif/Documents/STARIndex/Log.out ]]; then rm -v /home/leif/Documents/STARIndex/Log.out; fi

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Wed Dec 15 20:55:03 2021] rule hisat2Index: input: /home/leif/Documents/genome_fasta/genome.fa output: /home/leif/Documents/HISAT2Index/genome.6.ht2 log: logs/hisat2Index.log jobid: 9 threads: 5

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 ln: failed to create symbolic link '/home/leif/Documents/HISAT2Index/genome.fa': File exists [Wed Dec 15 20:55:03 2021] Error in rule hisat2Index: jobid: 9 output: /home/leif/Documents/HISAT2Index/genome.6.ht2 log: logs/hisat2Index.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    ln -s /home/leif/Documents/genome_fasta/genome.fa /home/leif/Documents/HISAT2Index/genome.fa
    hisat2-build -q -p 5 /home/leif/Documents/HISAT2Index/genome.fa /home/leif/Documents/HISAT2Index/genome
    2> logs/hisat2Index.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Wed Dec 15 20:55:03 2021] rule makeKnownSpliceSites: input: /home/leif/Documents/annotation/genes.gtf output: /home/leif/Documents/HISAT2Index/splice_sites.txt log: logs/makeKnownSpliceSites.log jobid: 10 threads: 5

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 [Wed Dec 15 20:55:03 2021] Error in rule makeKnownSpliceSites: jobid: 10 output: /home/leif/Documents/HISAT2Index/splice_sites.txt log: logs/makeKnownSpliceSites.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    hisat2_extract_splice_sites.py /home/leif/Documents/annotation/genes.gtf > /home/leif/Documents/HISAT2Index/splice_sites.txt 2> logs/makeKnownSpliceSites.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job makeKnownSpliceSites since they might be corrupted: /home/leif/Documents/HISAT2Index/splice_sites.txt Job failed, going on with independent jobs. Exiting because a job execution failed. Look above for error message Complete log: /home/leif/Documents/.snakemake/log/2021-12-15T205500.776584.snakemake.log

!!! ERROR in index creation workflow! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Error: snakemake returned an error code of 1, so processing is incomplete!

adRn-s commented 2 years ago

Could you try to do conda activate /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8? Please report full error message. We need more information than just the simple "it is not a conda environment". Also, tree /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 would be nice to check.

LeilyR commented 2 years ago

why did you need to export PATH="/path/to/miniconda3/bin:$PATH" ? I am afraid there is something wrong with your conda installation. try which conda to begin with. do you have other pythons on your system? Read the conda documentation on how to install and use conda.

Leif78 commented 2 years ago

thanks for input. @adRn-s full error message: (snakePipes) leif@leif-OptiPlex-9020:~$ conda activate /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Not a conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8

tree /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8

@LeilyR : I export PATH since it was explained here https://snakepipes.readthedocs.io/en/latest/content/setting_up.html

which conda /home/leif/miniconda3/bin/conda

I can try a new installation without changing path.

Leif78 commented 2 years ago

I reinstalled ubuntu 20.04. Installed miniconda3 installed snakePipes: conda create -n snakePipes -c mpi-ie -c conda-forge -c bioconda snakePipes==2.5.3

snakePipes createEnvs

(snakePipes) leif@leif-OptiPlex-9020:~/Documents$ createIndices -o ~/Documents/indices_GRCz11 --genomeURL ~/Documents/GRCz11/Danio_rerio.GRCz11.dna_sm.primary_assembly.fa --gtfURL ~/Documents/GRCz11/Danio.gtf --local GRCz11

---- This analysis has been done using snakePipes version 2.5.3 ---- Building DAG of jobs... Using shell: /usr/bin/bash Provided cores: 5 Rules claiming more threads will be scaled down. Job counts: count jobs 1 all 1 bowtie2Index 1 bwaIndex 1 bwamethIndex 1 computeEffectiveGenomeSize 1 createGenomeFasta 1 downloadGTF 1 extendGenicRegions 1 fastaDict 1 fastaIndex 1 gtf2BED 1 hisat2Index 1 make2bit 1 makeKnownSpliceSites 1 starIndex 15

[Thu Dec 16 15:20:00 2021] rule downloadGTF: output: /home/leif/Documents/indices_GRCz11/annotation/genes.gtf jobid: 5

[Thu Dec 16 15:20:00 2021] rule createGenomeFasta: output: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa jobid: 1

---- This analysis has been done using snakePipes version 2.5.3 ----

---- This analysis has been done using snakePipes version 2.5.3 ---- Job counts: count jobs 1 downloadGTF 1 Job counts: count jobs 1 createGenomeFasta 1 [Thu Dec 16 15:20:07 2021] Finished job 5. 1 of 15 steps (7%) done

[Thu Dec 16 15:20:07 2021] rule gtf2BED: input: /home/leif/Documents/indices_GRCz11/annotation/genes.gtf output: /home/leif/Documents/indices_GRCz11/annotation/genes.bed log: logs/gtf2BED.log jobid: 6

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 /usr/bin/bash: gtfToGenePred: command not found [Thu Dec 16 15:20:12 2021] Error in rule gtf2BED: jobid: 6 output: /home/leif/Documents/indices_GRCz11/annotation/genes.bed log: logs/gtf2BED.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    awk '{if ($3 != "gene") print $0;}' /home/leif/Documents/indices_GRCz11/annotation/genes.gtf             | grep -v "^#"             | gtfToGenePred /dev/stdin /dev/stdout             | genePredToBed stdin /home/leif/Documents/indices_GRCz11/annotation/genes.bed 2> logs/gtf2BED.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs. [Thu Dec 16 15:20:13 2021] Finished job 1. 2 of 15 steps (13%) done

[Thu Dec 16 15:20:13 2021] rule fastaIndex: input: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa output: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa.fai log: logs/fastaIndex.log jobid: 2

[Thu Dec 16 15:20:13 2021] rule fastaDict: input: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa output: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.dict log: logs/fastaDict.log jobid: 3

[Thu Dec 16 15:20:13 2021] rule computeEffectiveGenomeSize: input: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa output: /home/leif/Documents/indices_GRCz11/genome_fasta/effectiveSize log: logs/computeEffectiveGenomeSize.log jobid: 14

[Thu Dec 16 15:20:13 2021] rule bwaIndex: input: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa output: /home/leif/Documents/indices_GRCz11/BWAIndex/genome.fa.sa log: logs/bwaIndex.log jobid: 12

[Thu Dec 16 15:20:13 2021] rule bwamethIndex: input: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa output: /home/leif/Documents/indices_GRCz11/BWAmethIndex/genome.fa.bwameth.c2t.sa log: logs/bwamethIndex.log jobid: 13

Activating conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Activating conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Activating conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Not a conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Not a conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 /usr/bin/bash: seqtk: command not found [Thu Dec 16 15:20:15 2021] Error in rule fastaDict: [Thu Dec 16 15:20:15 2021] jobid: 3 Error in rule fastaIndex: output: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.dict jobid: 2 [Thu Dec 16 15:20:15 2021] log: logs/fastaDict.log (check log file(s) for error message) output: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa.fai Error in rule bwamethIndex: [Thu Dec 16 15:20:15 2021] conda-env: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8 log: logs/fastaIndex.log (check log file(s) for error message) jobid: 13 Error in rule computeEffectiveGenomeSize: shell:

    samtools dict -o /home/leif/Documents/indices_GRCz11/genome_fasta/genome.dict /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa 2> logs/fastaDict.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
conda-env: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8
output: /home/leif/Documents/indices_GRCz11/BWAmethIndex/genome.fa.bwameth.c2t.sa
jobid: 14

shell:

    samtools faidx /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa 2> logs/fastaIndex.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
log: logs/bwamethIndex.log (check log file(s) for error message)
output: /home/leif/Documents/indices_GRCz11/genome_fasta/effectiveSize

conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5

Job failed, going on with independent jobs. log: logs/computeEffectiveGenomeSize.log (check log file(s) for error message) shell:

    ln -s /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa /home/leif/Documents/indices_GRCz11/BWAmethIndex/genome.fa
    bwameth.py index /home/leif/Documents/indices_GRCz11/BWAmethIndex/genome.fa 2> logs/bwamethIndex.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
conda-env: /home/leif/miniconda3/envs/82d65abf560ab5d8bf505d707c817ea8

Job failed, going on with independent jobs.

shell:

    seqtk comp /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa | awk '{tot += $3 + $4 + $5 + $6}END{print tot}' > /home/leif/Documents/indices_GRCz11/genome_fasta/effectiveSize 2> logs/computeEffectiveGenomeSize.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

[Thu Dec 16 15:20:15 2021] Job failed, going on with independent jobs. rule make2bit: input: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa output: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.2bit log: logs/make2bit.log jobid: 4

Removing output files of failed job computeEffectiveGenomeSize since they might be corrupted: /home/leif/Documents/indices_GRCz11/genome_fasta/effectiveSize Job failed, going on with independent jobs. Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 [Thu Dec 16 15:20:16 2021] Error in rule bwaIndex: jobid: 12 output: /home/leif/Documents/indices_GRCz11/BWAIndex/genome.fa.sa log: logs/bwaIndex.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    ln -s /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa /home/leif/Documents/indices_GRCz11/BWAIndex/genome.fa
    bwa index /home/leif/Documents/indices_GRCz11/BWAIndex/genome.fa 2> logs/bwaIndex.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs. Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 [Thu Dec 16 15:20:17 2021] Error in rule make2bit: jobid: 4 output: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.2bit log: logs/make2bit.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    faToTwoBit /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa /home/leif/Documents/indices_GRCz11/genome_fasta/genome.2bit 2> logs/make2bit.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Thu Dec 16 15:20:17 2021] rule bowtie2Index: input: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa output: /home/leif/Documents/indices_GRCz11/BowtieIndex/genome.rev.2.bt2 log: logs/bowtie2Index.log jobid: 8 threads: 5

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 /usr/bin/bash: line 1: bowtie2-build: command not found [Thu Dec 16 15:20:20 2021] Error in rule bowtie2Index: jobid: 8 output: /home/leif/Documents/indices_GRCz11/BowtieIndex/genome.rev.2.bt2 log: logs/bowtie2Index.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    ln -s /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa /home/leif/Documents/indices_GRCz11/BowtieIndex/genome.fa
    bowtie2-build -t 5 /home/leif/Documents/indices_GRCz11/BowtieIndex/genome.fa /home/leif/Documents/indices_GRCz11/BowtieIndex/genome
    if [[ -f BowtieIndex/genome.rev.2.bt2l ]]; then ln -s genome.rev.2.bt2l /home/leif/Documents/indices_GRCz11/BowtieIndex/genome.rev.2.bt2 ; fi
    2> logs/bowtie2Index.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Thu Dec 16 15:20:20 2021] rule hisat2Index: input: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa output: /home/leif/Documents/indices_GRCz11/HISAT2Index/genome.6.ht2 log: logs/hisat2Index.log jobid: 9 threads: 5

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 /usr/bin/bash: line 1: hisat2-build: command not found [Thu Dec 16 15:20:22 2021] Error in rule hisat2Index: jobid: 9 output: /home/leif/Documents/indices_GRCz11/HISAT2Index/genome.6.ht2 log: logs/hisat2Index.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    ln -s /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa /home/leif/Documents/indices_GRCz11/HISAT2Index/genome.fa
    hisat2-build -q -p 5 /home/leif/Documents/indices_GRCz11/HISAT2Index/genome.fa /home/leif/Documents/indices_GRCz11/HISAT2Index/genome
    2> logs/hisat2Index.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Thu Dec 16 15:20:22 2021] rule starIndex: input: /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa output: /home/leif/Documents/indices_GRCz11/STARIndex/SAindex log: logs/starIndex.log jobid: 11 threads: 5

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 [Thu Dec 16 15:20:24 2021] Error in rule starIndex: jobid: 11 output: /home/leif/Documents/indices_GRCz11/STARIndex/SAindex log: logs/starIndex.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    STAR --runThreadN 5 --runMode genomeGenerate --genomeDir /home/leif/Documents/indices_GRCz11/STARIndex --genomeFastaFiles /home/leif/Documents/indices_GRCz11/genome_fasta/genome.fa 2> logs/starIndex.log
    if [[ -w Log.out ]]; then rm -v Log.out; elif [[ -w /home/leif/Documents/indices_GRCz11/STARIndex/Log.out ]]; then rm -v /home/leif/Documents/indices_GRCz11/STARIndex/Log.out; fi

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Job failed, going on with independent jobs.

[Thu Dec 16 15:20:24 2021] rule makeKnownSpliceSites: input: /home/leif/Documents/indices_GRCz11/annotation/genes.gtf output: /home/leif/Documents/indices_GRCz11/HISAT2Index/splice_sites.txt log: logs/makeKnownSpliceSites.log jobid: 10 threads: 5

Activating conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 Not a conda environment: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 [Thu Dec 16 15:20:26 2021] Error in rule makeKnownSpliceSites: jobid: 10 output: /home/leif/Documents/indices_GRCz11/HISAT2Index/splice_sites.txt log: logs/makeKnownSpliceSites.log (check log file(s) for error message) conda-env: /home/leif/miniconda3/envs/02934cc2df41b399b23179bb51fb19c5 shell:

    hisat2_extract_splice_sites.py /home/leif/Documents/indices_GRCz11/annotation/genes.gtf > /home/leif/Documents/indices_GRCz11/HISAT2Index/splice_sites.txt 2> logs/makeKnownSpliceSites.log

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job makeKnownSpliceSites since they might be corrupted: /home/leif/Documents/indices_GRCz11/HISAT2Index/splice_sites.txt Job failed, going on with independent jobs. Exiting because a job execution failed. Look above for error message Complete log: /home/leif/Documents/indices_GRCz11/.snakemake/log/2021-12-16T152000.798193.snakemake.log

!!! ERROR in index creation workflow! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Error: snakemake returned an error code of 1, so processing is incomplete!

LeilyR commented 2 years ago

well seems you have an empty folder to be called as env. So it is not going to be recognized as a conda env. Please check how snakePipes createEnvs worked.

Leif78 commented 2 years ago

indeed, it seems that all envs folders are empty.

snakePipes createEnvs :

Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/shared.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/createIndices.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/rna_seq.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/rMats.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/sc_rna_seq.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/sc_rna_seq_seurat3.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/sc_rna_seq_loompy.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/sc_rna_seq_alevinqc.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/sc_rna_seq_eisaR.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/dna_mapping.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/chip_seq.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/atac_seq.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/hic.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/wgbs.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/rmarkdown.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/preprocessing.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/noncoding.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/sambamba.yaml in /home/leif/miniconda3/envs Creating environment from /home/leif/miniconda3/envs/snakePipes/lib/python3.10/site-packages/snakePipes/shared/rules/envs/pysam.yaml in /home/leif/miniconda3/envs

adRn-s commented 2 years ago
  1. conda install mamba
  2. conda create -n snakePipes -c mpi-ie -c conda-forge -c bioconda snakePipes=2.5.3
  3. conda activate snakePipes
  4. snakePipes createEnvs

And you should see the process as they get installed. We followed the above steps and could not reproduce your issue.

katsikora commented 2 years ago

I'm closing this for lack of activity. Please reopen if needed. Best, Katarzyna