sciabola
commented
3 years ago Very interesting point. I think we’ve to think a bit to this. It is definitely a great direction to think about! Marco
Prof. Marco Gamba, PhD Dipartimento di Scienze della Vita e Biologia dei Sistemi - Life Sciences and Systems Biology Via Accademia Albertina, 13 10123 Torino, Italia mail: marco.gamba@unito.it tel: (+39) 011-670-4560/4559
On 1 Feb 2021, at 19:07, Ben Lansdell notifications@github.com wrote:
Hi,
I'm a data scientist at St Jude, working on classifying animal behavior (mice). I've been using deeplabcut http://www.mousemotorlab.org/deeplabcut to track the pose, before playing with some ML methods to classify behaviors from annotated videos. BORIS seems like a great tool to take care of the behavior labeling step of this process.
Building the ML classifier is obviously an iterative process -- label, train, inspect results, relabel if needed, etc. We're been thinking about how to make this as streamlined as possible. This means making the train-inspect-relabel loop as tight as possible. I wonder if BORIS could be built on in a way that would allow this?
This would mean something like supporting analysis plugins that could run some of the training code in the background. The ideal process would be something like: the labeling would be performed, training of the model run, the results overlaid in the video in BORIS, and perhaps frames in the video suggested to relabel, then retrain, etc. But to begin with, just something that can handle both the video labeling and ML training/inference, would be a useful start.
Is support for custom analysis plugins something you have thought about? Would this sort of capability be of interest if we were to develop something in this direction? Or, perhaps, do you have ideas/other tools we could use to about how achieve what I have described? (We've looked at simba, but the labeling process is much nicer in BORIS, so having something the works more closely with BORIS seems useful)
Thanks!
Regards, Ben Lansdell
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/olivierfriard/BORIS/issues/356, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABW7UQKZ3TG3BZGB3XTQSW3S43UX3ANCNFSM4W5HJ5RQ.
benlansdell
Hi,
I'm a data scientist at St Jude, working on classifying animal behavior (mice). I've been using deeplabcut to track the pose, before playing with some ML methods to classify behaviors from annotated videos. BORIS seems like a great tool to take care of the behavior labeling step of this process.
Building the ML classifier is obviously an iterative process -- label, train, inspect results, relabel if needed, etc. We're been thinking about how to make this as streamlined as possible. This means making the train-inspect-relabel loop as tight as possible. I wonder if BORIS could be built on in a way that would allow this?
This would mean something like supporting analysis plugins that could run some of the training code in the background. The ideal process would be something like: the labeling would be performed, training of the model run, the results overlaid in the video in BORIS, and perhaps frames in the video suggested to relabel, then retrain, etc. But to begin with, just something that can handle both the video labeling and ML training/inference, would be a useful start.
Is support for custom analysis plugins something you have thought about? Would this sort of capability be of interest if we were to develop something in this direction? Or, perhaps, do you have ideas/other tools we could use to about how achieve what I have described? (We've looked at simba, but the labeling process is nicer in BORIS, so having something the works more closely with BORIS seems useful)
Thanks!
Regards, Ben Lansdell