sronilsson
commented
1 year ago Hi @fon215! You are correct - the goal is definitly to not label behaviors in new videos and to create classifiers that can score behaviors on videos that you didn’t use for training.
The issue seems to be false negatives in the new videos: no scratches are detected. If you lower the discrimination threshold for the new videos, how does it look? Is there anything new you can think of about the new videos relative to training/previous validation videos - e.g., camera angles, recording environment?
fon215
Hello! I am using SimBA to do automated behavioral analysis of nape scratching and previously I trained the left and right nape scratching models on 20 videos, but as a part of my research we would like to see how the model performs on new videos with nape scratching, running the same model on new videos. I noticed that the videos that I labeled behavior on and produced machine results of in the past worked but the 7 new videos output machine result files with no scratches (1s) recorded even though the DLC tracking performed well on those videos and it is clearly visible to the human eye that scratching is occurring. I tried validating the model on a previous video with a 0.70 threshold and 100ms minimum bout length and that looked like it gave good results but when I validate on a single video on one of the new added videos it still does not classify any scratches. From my understanding, you should not have to label behavior if you add new videos to the project but instead go to the Run Machine Model step after you have imported tracking data and videos, set video parameters, ran outlier correction, and extracted features on these new videos. I would like to use existing models for left and right nape scratching but it does not seen to be performing correctly. I was wondering if you had any advice on how to fix this problem and if I am missing something or if there is an error somewhere. I took some screenshots of what my folders look like here:
Thanks so much!