search

nanoporetech / ont_fast5_api

Oxford Nanopore Technologies fast5 API software
Other
147 stars 29 forks source link

fast5_subset OSError #62

Closed ndierckx closed 3 years ago

ndierckx commented 3 years ago

HI,

I was trying the fast5_subset option, but I always get an error, I tried on 3 different fast5 files of one run

`DEBUG:h5py._conv:Creating converter from 5 to 3 | 0% ETA: --:--:-- Traceback (most recent call last): File "/home/nicolas/miniconda3/lib/python3.7/site-packages/ont_fast5_api/conversion_tools/conversion_utils.py", line 280, in extract_selected_reads for read_id, read in read_generator(input_file, read_set): File "/home/nicolas/miniconda3/lib/python3.7/site-packages/ont_fast5_api/conversion_tools/conversion_utils.py", line 312, in read_generator with get_fast5_file(str(input_file)) as input_f5: File "/home/nicolas/miniconda3/lib/python3.7/site-packages/ont_fast5_api/fast5_interface.py", line 12, in get_fast5_file if is_multi_read(filepath): File "/home/nicolas/miniconda3/lib/python3.7/site-packages/ont_fast5_api/fast5_interface.py", line 40, in is_multi_read with MultiFast5File(filepath, mode='r') as f5_file: File "/home/nicolas/miniconda3/lib/python3.7/site-packages/ont_fast5_api/multi_fast5.py", line 15, in init self.handle = h5py.File(self.filename, self.mode, driver=driver) File "/home/nicolas/miniconda3/lib/python3.7/site-packages/h5py/_hl/files.py", line 458, in init swmr=swmr) File "/home/nicolas/miniconda3/lib/python3.7/site-packages/h5py/_hl/files.py", line 199, in make_fid fid = h5f.open(name, flags, fapl=fapl) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "h5py/h5f.pyx", line 100, in h5py.h5f.open OSError: Unable to open file (truncated file: eof = 134217728, sblock->base_addr = 0, stored_eof = 702883507)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/nicolas/miniconda3/bin/fast5_subset", line 8, in sys.exit(main()) File "/home/nicolas/miniconda3/lib/python3.7/site-packages/ont_fast5_api/conversion_tools/fast5_subset.py", line 167, in main multifilter.run_batch() File "/home/nicolas/miniconda3/lib/python3.7/site-packages/ont_fast5_api/conversion_tools/fast5_subset.py", line 81, in run_batch self.worker.run_batch(pool=None) File "/home/nicolas/miniconda3/lib/python3.7/site-packages/ont_fast5_api/conversion_tools/conversion_utils.py", line 172, in run_batch self._launch_sync_tasks() File "/home/nicolas/miniconda3/lib/python3.7/site-packages/ont_fast5_api/conversion_tools/conversion_utils.py", line 183, in _launch_sync_tasks reads, out_file, in_file = extract_selected_reads(*args_tuple) File "/home/nicolas/miniconda3/lib/python3.7/site-packages/ont_fast5_api/conversion_tools/conversion_utils.py", line 294, in extract_selected_reads raise ExtractionException(exception, output_file) ont_fast5_api.conversion_tools.conversion_utils.ExtractionException: (OSError("Error processing file /media/nicolas/Data/Mappings/Perl/fast5/PAG19221_pass_bb508309_403.fast5: ('Unable to open file (truncated file: eof = 134217728, sblock->base_addr = 0, stored_eof = 702883507)',)"), PosixPath('/media/nicolas/Data/Mappings/Perl/batch0.fast5'))`

fbrennen commented 3 years ago

Hi @ndierckx -- this looks to me like the fast5 files are corrupt. Can you read them with a tool like hdfview or h5dump?

ndierckx commented 3 years ago

Hi,

Thanks for the fast reply They open without problems with hdfview and h5dump

But I tried another run, and fast5_subset works with those files. So not sure why they got corrupted Is there a way to fix corrupted files?

fbrennen commented 3 years ago

Unfortunately not -- HDF5 is a fairly brittle format, and if a file gets corrupted there's basically nothing you can do. It sounds like you've solved your issue though!

ndierckx commented 3 years ago

OK thanks for the help!