Outlier correction - movement

[linab8]

Hello! I have a problem with outlier correction. The bodyparts I used in outlier correction were "tip of the nose" and "tail base". Whichever value I put as "Movement criterion", I always end up with all 253 videos corrected. The percentage of correction is 1 in all cases. I've tried values of 1, 2, and even 3. My frame rate is 25 frames per second - could that be the problem?

1. Choose bodyparts "nose tip" and "tail base" for outlier correction
2. Set "Movement criterion" to 1-3
3. Everything is corrected

[sronilsson]

Hi @linab8! Just so I understand what is happening: is it that all of your body-parts, in all videos, are deemed an outlier?

What happens if you set the outlier criterion to some super high value, like 10000?

[linab8]

Yes, all body-parts in all videos are labelled as outliers. I'll try that now, but I wasn't sure how high I should go since it's recommended 0.7 - 1.5 and I already used a larger value. @sronilsson

[sronilsson]

Yeah I was just suggesting the large value for troubleshooting, to figure out what the error may, if it's unrelated to what value you put in

[linab8]

So, I tried with 10000 as a Movement criterion value and it still deemed all body parts in all videos as outliers. @sronilsson Do you have any other idea?

[sronilsson]

I am not sure, I need some more information. Do you have the output file with the outlier scores you can show me where you see that all the values are deemed outliers?

Also, if possible, could you share me a very small project to see if I can reproduce your error? Say sharing your project through a gdrive but without any videos, and only with a single file inside the project_folder/csv/input_csv directory?

[sronilsson]

Hi again @linab8 - I've played around a little trying to recreate the issue you're seeing.

In short, I could recreate something very similar to what you are seeing if the [Outlier settings] part of your SimBA project_config.ini contains multiple different settings for each animal (the section in the screengrab below). I will insert a code to prevent this from happening, and let you know when done.

[sronilsson]

@linab8

If you update simba using pip install simba-uw-tf-dev --upgrade, and try again by setting the body-parts to use to correct outliers - do you still see all the body-parts being corrected?

What I did was to insert a potential fix which removes any previous information on which body-parts to use inside the project_config, before inserting the body-parts you set in the menu.

[linab8]

Thank you for your help. I've updated Simba and am now waiting for the results of outlier correction. Here's what my results looked like yesterday. The first picture is Movement, and the second is Location. @sronilsson

[linab8]

Unofortunately, I'm getting the same results even with the updated Simba. @sronilsson

[sronilsson]

Got it, thanks for testing @linab8. Could you paste me your project_config.ini file here and I can check what is going on? I think you can drag it into the box. You may have to zip it first.

[linab8]

linab_simba.zip Here's the zipped folder with the config.ini inside. Thank you for still working on this with me!

[sronilsson]

That looks like it should..

One thing that comes to mind is if "glave" and "repa" are not the body-parts you think they are. Say that the "glave" columns in your data is actually the data of the left ear, and "repa" is the data for your nose. Then SimBA would look for movement outliers using a criterion of 1.5x the mean left ear <-> nose distance which is small.. When you created your body-part configuration in SimBA, and listed all the body-parts, did you do it in the same order as they were annotated in the pose-estimation package?

[linab8]

Yes, but I don't have "left ear" or "right ear", but rather "beginning of the neck" and "end of the neck" since I don't work on mice, but I do have tip of the nose and tail base as the 1st and 8th point. I used the custom user defined body parts option. @sronilsson

[linab8]

I'll try once more with my 3rd point and 8th point (rather than 1st and 8th), in the case Simba recorded it in the order like it does with mice so it might read the 3d point as the tip of the nose. However, it was clear on my tracking model image that the "tip of the nose" is the 1st point. @sronilsson

[sronilsson]

Perhaps we can go back a few steps, I'd like to figure this one out....

Could you share me a very small project to see if I can reproduce your error? Say sharing your project through a gdrive? I know your project is very large, so I am just looking for a single video inside project_folder/videos, and a single imported file inside the project_folder/csv/input_csv directory, and no files needed inside the project_folder/csv/outlier_corrected_location or project_folder/csv/outlier_corrected_movement_location directories.

PS. you could try and click skip outlier correction and move on from this (but there might be some related errors downstream).

[linab8]

Okay, here's the link to the drive. I can create the new small project after another attempt of outlier correction finishes so I can close that project for a moment. @sronilsson https://drive.google.com/drive/folders/1N3EFNkCKtmzxnQ3KklQO6RGImw-mY7q5?usp=sharing I'd rather not skip outlier correction if possible since it was clear from the "Location" correction that some videos needed it.

[sronilsson]

Thank you @linab8 - have to step away from computer but will check later and let you know.

[sronilsson]

Oh, can I also get the project_folder/logs folder ?

[linab8]

No problem, I have to step away too. I've imported the Logs folder. @sronilsson

[sronilsson]

One @linab8 ! One more question - when you visualize your data in DLC, how does the tracking look like (without using any filtering)?

I am asking since I opened the AGGRESSIVITY_4_11_21_Trial_2_camera1.csv file. Looking at the first two columns (representing the X and Y coordinate of the first body-part) is largely static for the first several hundred frames, other body-parts show the same static integers for many many frames.

My guess is this: A lot of the body-part predictions are missing in your DLC data. I.e., in more than half of the frames, there is at least one and typically more body-part that your DeepLabCut model could not find (and gives it a None or NaN value). You then performed nearest interpolation in SimBA, to get body-part prediction in all frames. Although interpolation can be a solution, the interpolation will suffer when so many body-parts are missing predictions for extended time. Next, SimBA tries to calculate the mean distance between "glave" and "repa" in the video using these (predominantly) interpolated values. However, most of these are not accurate, and the outlier correction fails.

[linab8]

There are some problems when visualizing data - some ID swaps exist even in that first video, however, we thought that we'd smooth that with outlier correction in Simba. Because after evaluating network in DLC, I retrained it with additional frames and it showed better results. Do you suggest more training in DLC? I understand what you're saying and it does make sense. I just didn't got the impression that more than half of the frames in my videos have some body part prediction missing. Would better tracking data (less ID swaps or missing body parts) fix what I'm experience in Simba outlier correction? @sronilsson Thank you so much for looking into my project.

[sronilsson]

@linab8 yes, there are a few things in DLC that could potentially mask the performance and make it look better than it actually is, and watch out for these, there is more discussion HERE, I recommend getting good predictions on as much data as possible and more training (from what I have seen)

(i) DLC has a filtering setting (I don't know if it is used by default or if you used it). This sets all body-part predictions, where the confidence are below a certain user-specified threshold, no "None" and hides them. If you have used it, that could explain why you have so many 'None' values that SimBA has to interpolate. I would not recommend using filtering.

(ii) I have sometimes seen DLC plot either all animals in the same colors, or e.g., the left ear of one animal in the same color as the left ear of animal two. If there is an ID switch, and the left ear of animal one becomes the left ear of animal two, you won't notice because they are the same color and it masks incorrect performance of the tracking model.

[linab8]

I'll try some additional training and check that the filtering option is turned off - it might not be. Concerning colours, I used the option of colouring by individual so it's easy to see when ID is switched in some cases. Thank you for everything and I'll comment on this thread when I succeed or if I need additional help!