search

hasindu2008 / slow5tools

Slow5tools is a toolkit for converting (FAST5 <-> SLOW5), compressing, viewing, indexing and manipulating data in SLOW5 format.
https://hasindu2008.github.io/slow5tools
MIT License
94 stars 6 forks source link

Error in slow5tools merge #112

Closed lyj95618 closed 3 months ago

lyj95618 commented 4 months ago

Hi,

I tried to use fast5toslow5 to convert the fast5 files to slow5 and then merge them into a single file. I am getting the following error during the merge step

[list_all_items] Looking for '*.fast5' files in /hpf/projects/bkalish/MIA_m6A/nanopore/new_samples/raw_data/Sample1
[list_all_items] Looking for '*.fast5' files in /hpf/projects/bkalish/MIA_m6A/nanopore/new_samples/raw_data/Sample1/fast5
[f2s_main] 2747 fast5 files found - took 1.053s
[f2s_main] Just before forking, peak RAM = 0.000 GB
[f2s_iop] 8 proceses will be used.
[f2s_iop] Spawning 8 I/O processes to circumvent HDF hell.
[f2s_child_worker::ERROR] Bad fast5: Fast5 file '/hpf/projects/bkalish/MIA_m6A/nanopore/new_samples/raw_data/Sample1/fast5/PAW00730_6e7a5de0_3b3dcc69_14.82_4.fast5' could not be opened or is corrupted.
[f2s_child_worker::ERROR] Bad fast5: Fast5 file '/hpf/projects/bkalish/MIA_m6A/nanopore/new_samples/raw_data/Sample1/fast5/PAW00730_6e7a5de0_3b3dcc69_3.67_5.fast5' could not be opened or is corrupted.
[f2s_iop] Child process 2794779 exited with status=1.
[list_all_items] Looking for '*.slow5' files in /hpf/largeprojects/ccmbio/acelik_files/kalish/nanopore/nanopore/lauren_test/debug_nanopolish/try_f5c/slow5_dir/Sample1
[merge_main] 158 files found - took 0.049s

[slow5_hdr_init::ERROR] Malformed blow5 header. Failed to read the magic number. EOF reached. At src/slow5.c:801
[slow5_init::ERROR] Parsing slow5 header of file '/hpf/largeprojects/ccmbio/acelik_files/kalish/nanopore/nanopore/lauren_test/debug_nanopolish/try_f5c/slow5_dir/Sample1/PAW00730_6e7a5de0_3b3dcc69_27.20_2.blow5' failed. At src/slow5.c:153
[slow5_open_with::ERROR] Exiting on error. At src/slow5.c:369

I have slow5 files output but the merged.blow5 file is empty.

I am using the most recent binary version of slow5tools. The original fast5 files are converted from the pod5 files with the pod5 program. The basecaller is dorado.

Thanks! Laur

hasindu2008 commented 4 months ago

The error suggests that the input fast5 files to slow5tools are not in the expected schema. Could you try directly converting the pod5 files directly to blow5 using https://github.com/Psy-Fer/blue-crab.

I have a feeling that the pod5 to fast5 converter could be creating fast5 in an unexpected fashion which the slow5tools cannot recognise. Let's see if the direct conversion works.

lyj95618 commented 4 months ago

Thank you so much for the quick response! I will get the pod5 files and try the blue-crab. Will keep you posted! Thanks a lot!

Laur

hasindu2008 commented 3 months ago

@lyj95618 Has this problem been sorted out?

lyj95618 commented 3 months ago

Yes thanks! Everything was solved after using the original pod5 files with blue-crab. Thank you for all the helps!