search

talmolab / sleap

A deep learning framework for multi-animal pose tracking.
https://sleap.ai
Other
435 stars 97 forks source link

Import analysis HDF5 results in 1's for all instance confidence scores. #1466

Open calebvogt opened 1 year ago

calebvogt commented 1 year ago

Windows 10 Sleap v1.3.0

Issue: Exported .h5 files using the File > Export Analysis HDF5 > all videos functionality yields instances which have uniform confidence scores of 1, despite the fact that within the actual project the predicted instances show variable scores (fluctuating between 0-1.02).

Details: Top-down multi-animal model for tracking a single mouse.

Screenshot #1: image

The above screenshot shows what the prediction score looks like for an instance within a project. Instance scores fluctuate across frames as expected. I then export the .h5 file using the sleap GUI as described above.

When I import the exported .h5 file and associated video into a new SLEAP window, I get the following: image

The instances across the entire file now are locked at 1.0. Is it possible that they are actually being exported properly and are just not showing up when they are loaded into the sleap gui? I have not yet attempted to open up the .h5 files and explore the structure to see if the scores are hidden away somewhere.

The next step in my pipeline is to load these .h5 files into Simba for further analysis, for what its worth.

Thanks in advance for your help!!! Happy to provide further details.

Edit #1: I inspected one of the exported .h5 files using the R rhdf5 package. I can see that the group elements in the .h5 file are as shown below: image

Opening up the instance_scores group yields the following: image

So it seems like the prediction scores ARE exported properly, but for some reason when the file is reimported into a sleap GUI, they are shown uniformly as 1. This isnt a major issue fundamentally, but if I were running inference in batch across hundreds of videos via the command line, and then wanting to inspect .h5 files individually using the sleap gui, I wouldnt be able to accurately assess the tracking confidence scores via the GUI. So minor bug?

For anyone who comes across this issue in the future, see this related issue regarding getting the confidence scores and why confidence values sometimes are greater than 1: https://github.com/talmolab/sleap/issues/521

While the confidence score in the GUI is still a minor issue, as long as I can access the confidence data I should be ok, so this issue can likely be closed.

Thanks!

roomrys commented 1 year ago

Hi @calebvogt,

While there is an import for the analysis, not all the info is there to fully recreate the slp (such as edges). I stand corrected, we include edge_names and edge_inds in the data_dict which creates the .analysis.h5. But the confidence scores is definitely something we can and should populate. I'll leave this open.

Thanks, Liezl

Tagging some more issues/discussion for improving the analysis to slp conversion: