Open martenson opened 5 years ago
I went through all of the above, the latest versions of the following tools support gz input:
this leaves us with
4056 | toolshed.g2.bx.psu.edu/repos/nate/trinity_psc/trinity_psc/0.0.1
3850 | toolshed.g2.bx.psu.edu/repos/lparsons/fastq_join/fastq_join/1.1.2-806.1
3600 | toolshed.g2.bx.psu.edu/repos/devteam/tophat2/tophat2/2.1.1
3329 | toolshed.g2.bx.psu.edu/repos/iuc/pear/iuc_pear/0.9.6.1
3074 | toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/bwa_wrapper/1.2.3
2827 | toolshed.g2.bx.psu.edu/repos/devteam/bowtie_wrappers/bowtie_wrapper/1.1.2
1536 | toolshed.g2.bx.psu.edu/repos/iuc/mothur_make_contigs/mothur_make_contigs/1.39.5.0
1228 | toolshed.g2.bx.psu.edu/repos/devteam/tophat2/tophat2/0.6
772 | toolshed.g2.bx.psu.edu/repos/iuc/flash/flash/1.2.11.3
588 | toolshed.g2.bx.psu.edu/repos/devteam/tophat2/tophat2/2.1.0
490 | toolshed.g2.bx.psu.edu/repos/devteam/kraken/kraken/1.2.3
436 | toolshed.g2.bx.psu.edu/repos/devteam/fastq_trimmer/fastq_trimmer/1.0.0
419 | toolshed.g2.bx.psu.edu/repos/devteam/picard/picard_FastqToSam/2.18.2.1
361 | toolshed.g2.bx.psu.edu/repos/devteam/tophat2/tophat2/0.9
243 | toolshed.g2.bx.psu.edu/repos/devteam/fastx_collapser/cshl_fastx_collapser/1.0.0
241 | toolshed.g2.bx.psu.edu/repos/iuc/kallisto_pseudo/kallisto_pseudo/0.43.1.1
I have added 'deprecated' tags to bwa_wrapper,tophat2 and pear is now hidden, this leaves us with
4056 | toolshed.g2.bx.psu.edu/repos/nate/trinity_psc/trinity_psc/0.0.1
3850 | toolshed.g2.bx.psu.edu/repos/lparsons/fastq_join/fastq_join/1.1.2-806.1
2827 | toolshed.g2.bx.psu.edu/repos/devteam/bowtie_wrappers/bowtie_wrapper/1.1.2
1536 | toolshed.g2.bx.psu.edu/repos/iuc/mothur_make_contigs/mothur_make_contigs/1.39.5.0
772 | toolshed.g2.bx.psu.edu/repos/iuc/flash/flash/1.2.11.3
490 | toolshed.g2.bx.psu.edu/repos/devteam/kraken/kraken/1.2.3
436 | toolshed.g2.bx.psu.edu/repos/devteam/fastq_trimmer/fastq_trimmer/1.0.0
419 | toolshed.g2.bx.psu.edu/repos/devteam/picard/picard_FastqToSam/2.18.2.1
243 | toolshed.g2.bx.psu.edu/repos/devteam/fastx_collapser/cshl_fastx_collapser/1.0.0
241 | toolshed.g2.bx.psu.edu/repos/iuc/kallisto_pseudo/kallisto_pseudo/0.43.1.1
I think this is quite important since the implicitly uncompressed files unnecessarily take space in user histories.
I started
Then there are 83 tools left in IUC:
egrep 'type="data"' tools -r --include "*xml" | grep fastq | grep -v ".gz" | sort -r | cut -d":" -f1 | uniq | wc -l
Feel free to reorder the tools by importance or put your name / PR-link next to it if you are working on it.
One potential problem that I just observer while treating minimap: Cannot index files compressed with gzip, please use bgzip
For this tool test I can simply use bgzip to generate the test data .. but how can we ensure this in the wild?
Maybe for minimap we need to unzip the files on the fly (even if the tool supports zipped input natively).
@bernt-matthias would you like me to take any of these in any particular order?
@gregvonkuster help is very welcome .. choose any :) and put your name or a PR link behind it
The first column is a number of times the given tool triggered implicit conversion (decompression) of a compressed fastq dataset in last ~6 months on Main.
If we could ensure the tools can work with compressed fastq files we would save a lot of quota for the users of these tools.
edit: added the full list of top converting causes