search

nanoporetech / ont_fast5_api

Oxford Nanopore Technologies fast5 API software
Other
149 stars 29 forks source link

feature request : add tar as input format for single_to_multi_fast5 #2

Closed svennd closed 5 years ago

svennd commented 5 years ago

Would it be possible to add .tar as input format for single_to_multi_fast5 ? I believe allot of customers that store fast5's have already made tars of these to save the filesystem from a inode/rsync hangover.

fbrennen commented 5 years ago

Hi @svennd -- other than ease of use, is there a particular reason you don't want to just untar each file first before converting it?

svennd commented 5 years ago

Its mainly the ease of use; But wear & tear of writing the same data two times could be a good reason. tar -> single fast5's -> multi fast5. If this tool can load 4k reads in memory from a tar, no write would be required to store the temporary single fast5's. Considering the current raw data size per run (~2TB) that's a big win for those poor disks ;-)

Site specific for us, we store raw data on tape, so if we could directly get a valid format from our tape that would be a good thing.

fbrennen commented 5 years ago

Sure, ok, we can think about that. I believe it will be a non-trivial addition so it's likely to take a while to make it up the priority list.