search

BeeCSI-Microbiome / taxonomic_profiling_pipeline

0 stars 1 forks source link

Minimize bowtie2 stdout #11

Open LLansing opened 2 years ago

LLansing commented 2 years ago

Standard out from each job is currently saved as files in the form snakejob...sh.o The .o files from bowtie2 are enormous (90GB+). I don't believe the contents are necessary.

A potential solution is using the --quiet flag in the bowtie2 command, but maybe there is a snakemake-sided solution

Kurtj-hub commented 2 years ago

Snakefile with the following shell command in rule bowtie2 (notice "--quiet" added) still has large .o files.

'(bowtie2 --quiet -p {threads} -x phiX -1 {input.fwd} -2 {input.rev} --un-conc-gz {params.out}/unmapped/{wildcards.sample}_R%_unmapped.fastq.gz) 2> {log}'

Will work on an alternative or investigate more about why "--quiet" isn't working as expected. Maybe a "clean-up" rule or appending a simple shell command somewhere else would be a good idea?

LLansing commented 2 years ago

Good to know. It's possible that the --quiet flag does not do what I had hoped it would do. Implementing a clean-up step is something I've heard of for snakemake and I a good thing to look into