search

sgoldenlab / simba

SimBA (Simple Behavioral Analysis), a pipeline and GUI for developing supervised behavioral classifiers
https://simba-uw-tf-dev.readthedocs.io/
GNU General Public License v3.0
273 stars 137 forks source link

Reorder Nodes using SLEAP Skeleton as input #241

Open herbertzhengwu opened 1 year ago

herbertzhengwu commented 1 year ago

Hi SimBA developers, is there an easy way to rearrange the order of the nodes when the skeleton generated by SLEAP is used as input for SimBA?

We are trying to feed the SLEAP pose-estimation data into SimBA to pull out behavioral syllables. The nodes have to be ordered in a specific way for this to work. In the SLEAP output, node 1 is nose, node 2 is L ear, node 3 is R ear, and node 4 is tail. I'd like to rearrange it to feed into SimBA so that node 1 is L ear, node 2 is R ear, node 3 is nose, and node 4 is tail. There seems to be such a feature for deeplabcut but could you please add this for SLEAP as well?

We can't change the node order in our best-performing SLEAP mode, but the order of this skeleton won't work with SimBa: so we are stuck. Thanks so much for your help!

sronilsson commented 1 year ago

Hi @herbertzhengwu! Yes I was asked for a similar function the other day and will try to insert something..

What was holding me back was this: The DLC body-part re-organize function takes e.g., the H5 file, reorganizes the columns in the only dataframe in the H5, and then re-saves the H5 - easy. The SLEAP SLP's and H5 files however, are more complex, and not so easy to get into the original data-structure before re-saving.

ps. you can still use a user-defined body-part config in SimBA.

herbertzhengwu commented 1 year ago

Thanks a lot for your reply! We'll look into the user-defined config options. In the meantime, we'd really appreciate it if there's a function like that!

sronilsson commented 1 year ago

@herbertzhengwu - what would be easier to insert would by a function that reorganizes SLEAP CSV files. SLEAP outputs .SLP or .H5 files with data that are tricky to re-organize. However, SLEAP H5 files can be transformed into CSV format discussed HERE that can be imported into SimBA. I can insert a function that reorganizes those CSV files before import, but not sure that would be useful in your case?

samaralsanti commented 2 weeks ago

Hi! Has a solution to this been developed? I'm facing a similar issue with my project, and it seems like the user-defined configuration isn't viable for functions like the directionality options.

sronilsson commented 1 week ago

Thanks for bumping this @samaralsanti - this slipped and hasn't been done but should be.

The re-order nodes function we have only works for DLC because DLC data is organized column-wise and only contains a single dataframe. SLEAP data files contains much more data than a single dataframe, and/or is organized row-wise, so needs different operation.

Would you be able to share a sample of your SLEAP data files with me and I can use that while testing / typing something up?

samaralsanti commented 1 week ago

Hi! Thanks for your quick response. I would be happy to share some data files. What is the best way to share those with you?

Also, I don't know if this is helpful for you, but while looking for a solution to this on the SLEAP discussion forum, I also found this response from the team over there. They seemed open to providing assistance with integrating this function in software like SimBA.

sronilsson commented 1 week ago

Thanks @samaralsanti thats cool, I didn't see those functions before. It seems to accept an SLP file. Perhaps we can try to use those as a function call in SimBA, and place it in a loop over all data files in a directory.

If not too large, could you share it through a gdrive? It would be best to share sleap SLP files rather than SLEAP CSV/H5, so we can try those functions out that you linked.