search

Psy-Fer / blue-crab

crab go snap snap
MIT License
24 stars 2 forks source link

[ERROR]: pore_type is 'R10.4.1' expected to be 'not_set'. #12

Open lborcard opened 3 weeks ago

lborcard commented 3 weeks ago

Dear Blue-crab team,

I ran into this error is there a fix for that or not yet?

03-Jul-24 15:44:55 - blue-crab - [ERROR]: pore_type is 'R10.4.1' expected to be 'not_set'. Please contact developers with this message.

thanks

Psy-Fer commented 3 weeks ago

Hey,

Ahh that message was left as a land mine. This was a field that pod5 has but wasn't using.

I put that message in to detect if they started actually using it. Looks like they did. Until they did I couldn't check formatting or types.

Now I can.

Can you give me more information about your pod5 files? Are they recent?

Cheers,

James

lborcard commented 3 weeks ago

Hi,

Yes they were generated today and could be basecalled using dorado 0.7.2 . What other info would you like?

best,

Loïc

Psy-Fer commented 3 weeks ago

Does the conversion actually finish or does it crash with that error?

It's midnight here, so I'll fix it tomorrow. I imagine data we produced today should also have this metadata.

James

lborcard commented 3 weeks ago

Ok thanks a lot for the swift response, it finished but the blow5 are empty. You dont use it on your own data? I am trying to use it to extract certain info(time) and out of curiosity as well.

lborcard commented 3 weeks ago

that's the message at the end of the process

03-Jul-24 16:11:55 - blue-crab - [ERROR]: pore_type is 'R10.4.1' expected to be 'not_set'. Please contact developers with this message.
03-Jul-24 16:11:55 - blue-crab - [ERROR]: Error encounted, exiting.
03-Jul-24 16:11:55 - blue-crab - [INFO]: pod5 -> s/blow5 complete
Psy-Fer commented 3 weeks ago

Yea we use it on all of our data

But could be because we use older versions of minknow that we haven't hit the new metadata field.

I'll fix this tomorrow.

I'll let you know when it's done

Psy-Fer commented 3 weeks ago

Hey,

Could you try the latest release, v0.1.1?

Cheers, James

lborcard commented 3 weeks ago

dear James,

thank you for the update, I just tried the newer version. I am getting an error related to the pyarrow package: trace

   len(batch.signal[batch_row_index].as_buffer()),
AttributeError: 'pyarrow.lib.LargeListScalar' object has no attribute 'as_buffer'
Psy-Fer commented 3 weeks ago

Which version of pod5 are you using? Make sure it's the latest with the latest numpy 2.0.0. there were some updates recently based on the new numpy release. Maybe try a fresh environment if that doesn't work.

That should work. Let me know how it goes James

lborcard commented 3 weeks ago

I installed the package in a new conda env after pulling the newest version. I will update pod5 and numpy and let you now.

lborcard commented 3 weeks ago

I can already tell you that numpy 2.0 is not compatible with the installed version of minknow api (v5.9.5) and pod5 version seems to be the latest (v0.3.11). Should I update pyarrow?

lborcard commented 3 weeks ago

I checked the docs of pyarrow for the doc and there is indeed no method as.buffer but I might be looking at the wrong docs.

Psy-Fer commented 3 weeks ago

I would advise you create a separate environment for blue crab.

Using python3 -m venv ./blue-crab source blue-crab/bin/activate

Then update pip and install the latest release by cloning the repository, moving into the directory and running pip install .

Rather than trying to run it in the same environment as other nanopore things or in conda which is notorious for screwing up dependency graphs.

James

lborcard commented 3 weeks ago

It was installed in a new micromamba env. I tried your method but somehow I am getting the same error. thank you for your help.

Psy-Fer commented 3 weeks ago

Could you post the results of. pip list

James

lborcard commented 3 weeks ago 

Package         Version
--------------- -------
blue-crab       0.1.1
h5py            3.11.0
iso8601         2.1.0
lib_pod5        0.3.11
more-itertools  10.3.0
numpy           2.0.0
packaging       24.1
pip             24.1.1
pod5            0.3.11
polars          0.20.31
pyarrow         16.1.0
pyslow5         1.2.0
pytz            2024.1
setuptools      59.6.0
tqdm            4.66.4
vbz-h5py-plugin 1.0.1
Psy-Fer commented 3 weeks ago

Hmmm yea okay.

You might need to downgrade pod5 to the previous version along with a numpy downgrade of 1. Rather than 2.

Let me know if you want some exact commands to do this if you run into trouble.

This seems to be a pod5 issue. See if the downgrade works and I'll see what the author of pod5 has to say about the error.

James