Bad fast5 - [fast5_attribute_itr::ERROR]

chilampoon commented 2 years ago

Hello, it's me again! So we have a new minion run using the updated minknow, then when I usedslow5tools to convert the fast5 files I just got these errors:

[list_all_items] Looking for '*.fast5' files in FAS16606_pass_13cc46f5_0.fast5
[f2s_main] 1 fast5 files found - took 0.000s
[f2s_main] Just before forking, peak RAM = 0.000 GB
[f2s_iop] 1 proceses will be used.
[fast5_attribute_itr::ERROR] Bad fast5: Attribute Raw/num_minknow_events in FAS16606_pass_13cc46f5_0.fast5 is unexpected. Please report this with an example FAST5 file at 'https://github.com/hasindu2008/slow5tools/issues' for us to investigate.
[read_fast5::ERROR] Bad fast5: Could not iterate over the read groups in the fast5 file FAS16606_pass_13cc46f5_0.fast5.
[f2s_child_worker::ERROR] Could not read contents of the fast5 file 'FAS16606_pass_13cc46f5_0.fast5'.

I don't know if it's caused by the new minknow or what, but I can use h5py to open them. Could you help fix this? Thanks.

I can also send over our fast5 if you need it.

chilampoon commented 2 years ago

here is the link of one of my fast5s: https://drive.google.com/file/d/135BicPDTIUjx3v_EUhx13puZAhTKrlZJ/view?usp=sharing

Psy-Fer commented 2 years ago

Hey Chilampoon,

We also encountered this value. Something ONT added to fast5, with a small announcement, in patch notes, with no impact statement of what it is, what it does, or how it impacts the fast5 spec. Another reason we made slow5...

Anyway...

A fix for this is on the dev branch. https://github.com/hasindu2008/slow5tools/commit/7852a24f2e9e775fd2e6d1f879ba8f77212643b9

You can either wait for us to cut a new version which will incorporate this fix, or you can build against the dev branch if you have an urgent need right now. However, some things are still finalising on the dev branch (though they are passing our testing), so fair warning.

I'm sure @hasindu2008 can give more info on when the next release is out with this fix.

Cheers, James F.

hasindu2008 commented 2 years ago

Oh, I had tested this on like 10 samples. But forgot to make a release. Just created a release now. Please check.

hasindu2008 commented 2 years ago

@chilampoon Did the latest release work for your FAST5s?

chilampoon commented 2 years ago

Oh yeah, actually before the release I've built slow5tools using the dev branch as James said and it worked well -- you guys are awesome, thank you!

@chilampoon Did the latest release work for your FAST5s?

hasindu2008 / slow5tools

Bad fast5 - [fast5_attribute_itr::ERROR] #81