berman-lab / ymap

YMAP - Yeast Mapping Analysis Pipeline : An online pipeline for the analysis of yeast genomic datasets.
MIT License
6 stars 6 forks source link

Consider using pipes instead of writing to disk #45

Open vladimirg opened 8 years ago

vladimirg commented 8 years ago

Third-party tools often read from and write to files on disk. This may introduce an I/O bottleneck. It may be possible to pipe input and output between tools, e.g. to PicardTools read group tool (https://www.biostars.org/p/56566/). It's only worth it if the performance improvement will be appreciable and it will not interfere with tools that are parallelized (e.g., the parallelized mpileup assumes the BAM input is in its final form and contains all chromosomes).