pickle.UnpicklingError: invalid load key, ','. When Running Model

[autumnca]

When I select Run Model under the Validate Model on a Single Video section of the Run machine model tab, I see the following in the SimBA terminal window:

But the following in my anaconda terminal:

And the SimBA terminal window does not change or give me any indication that the model is finished running after 48 hours (I am only running one 3-hour video for testing purposes). Any help on what I could do about this error or why this might be happening would be greatly appreciated!

[sronilsson]

Hi @autumnca !

No, this is an error happening early, when SimBA is trying to open the .sav format model file. The file is either not the correct file specified in the Select model file entry box (e.g., it is some other non-model file), or it is somehow corrupted. In this below entry box, did you select the an sav file? How big is the sav file in mb or kb that you selected?

[autumnca]

Thank you! I fixed the file I had under Select model file, selected Run Model, and received the following error in the SimBA terminal:

When I try to validate, I receive the following error in my anaconda terminal:

I tried just completely starting over, thinking maybe I had done something incorrectly in the beginning stages of my last SimBA project, but have now ran into the same errors.

[sronilsson]

Hello @autumnca!

This error is produced when SimBA is looking for the probabilities that the behavior is present in each frame following classification. SimBA expects two values to be returned (the probability that the behavior is absent, and the probability that the behavior is present). However, in this case, SimBA has one value returned (the probability that the behavior is absent) and it breaks because of it. This could happen if the classifier only had one type of annotations to work with to begin with (behavior absent) and no annotations of behavior present. Did you annotate the videos going into SimBA using the annotation interface?

[autumnca]

Okay, I annotated using BORIS. And I did make sure that the format of the BORIS csv file matched what is listed on the GitHub page.

[sronilsson]

Oh yes! :) To check that you have appended the annotations appropriately, you can check the files representing the videos inside the project-folder/csv/targets_inserted. There will be a column, towards the end, with the name of your classifier. Do you see any 1 in that column?

[autumnca]

Looking at that file, I only see 0s in that column.

[sronilsson]

I see. that means that although the file was created, the annotations where not appended correctly from your BORIS file. I can check the file to see what is going on. Do I have access to the latest files on the gdrive?

[autumnca]

I actually noticed one error in the BORIS file when double checking it, then I re-imported the BORIS annotations and now am seeing both 1s and 0s. I re-trained the machine model and just tried running the model and am now running into the following error in my anaconda terminal after clicking on Validate:

And thank you again for your help!!

[sronilsson]

@autumnca - almost there! let me look over this, I was just working on tidying this script up. I will get back to you when done, might take a day or two though if that is OK?

[sronilsson]

@autumnca - didn't take as long as expected! Could you update to simba 0.92.7 with pip install simba-uw-tf-dev --upgrade and let me know if that fixes it or not?

That said, this error happened because of a bug in how SimBA handles single animal tracking coming from SLEAP. I'm working my way through the code and repairing as I go along. However if you do come across errors, please let me know and I make priority.

[autumnca]

I just upgraded SimBA and it officially ran!! This is so exciting thank you so much for all of your help.

I did visualize the classification results on one video though, and the results were much less accurate than I anticipated. Mostly (1) the key points are not anywhere near the mouse for a very large portion of the video and (2) SimBA predicts no nesting the entire video. I'm not sure if this is a result of inaccurate measurements entered, not enough .slp data (although key points are tracked well in SLEAP), not enough labelled BORIS files, or an incorrectly imported file. Let me know if you have any recommendations or thoughts for this, otherwise I'll work on troubleshooting what this might be caused by.

[autumnca]

One more thing! I will add, the key points are actually very accurate only when the mouse hasn't been moving for at least 5 seconds or so, otherwise they are sometimes on the other side of the cage.

[sronilsson]

Thanks @autumnca - the body-part location prediction is odd, I didn't see it in my test projects. Can I test the code on your data, is it available on gdrive? Would you mind sending me the link again?

For the nesting predictions, does it help if you decrease the discrimination threshold?

[autumnca]

I just uploaded everything to the gdrive that's located here:

https://drive.google.com/drive/folders/19E19tdUvdG5ebwL9I8XbRgRc6_zkZZyG?usp=sharing

Let me know if there is anything else I should upload. And I tried decreasing the discrimination threshold and the program still didn't recognize nesting throughout the video.

[autumnca]

Just following up on this, but no rush. I know you're very busy and we just got through a holiday weekend. Thanks again!

[sronilsson]

Hey @autumnca! I'm on it and I have not forgotten, the day job took has taken over. I aim to troubleshoot your project first thing tomorrow.

[sronilsson]

Hi @autumnca! I had a look at the project. I began by looking at the file coming into SimBA from SLEAP. That file is saved inside the project_folder/csv/input_csv folder after you've imported the data. I opened it and I see a lot of 0. These are the frames where there were no body-part detected in SLEAP. I visualized the data in the project_folder/csv/input_csv/ using the tool documented HERE under the Visualize pose estimation in folder heading. And it does look a little wild, the visualization file is here (it is rather big). https://drive.google.com/file/d/13CL_N9B75Nm6cdWFzhXAcsEb-8ZXXwII/view?usp=sharing

We just need to confirm whats happened to the input data: I'm leaning towards that the SLP file that was inputted does not contain all the otherwise accurate predictions, could that be the case? Did the video you imported to SimBA not represent the video that was tracked in SLEAP?

[autumnca]

Definitely not every frame in the SLP file is accurate, but when I looked through them it seemed that most files were accurate or within reason. And I actually imported the same video to SimBA that I used to train SLEAP, just to test if the pose-estimation predictions provided by SLEAP would transition over to SimBA correctly.

[sronilsson]

Do you have a visualization coming out of SLEAP of the predictions, for this videom you could share?

[autumnca]

I just uploaded a video of the SLEAP predictions to the google drive I shared with you earlier. Let me know if that works or if that's not what you were looking for. Thanks again!

[sronilsson]

Hello @autumnca! I had a look at the video. It seems like, for the great majority of frames, one or body-parts are missing predictions. The jumping we see after importing the data into SImBA is from SimBA trying to interpolate the locations of the missing predictions, but it is not going very well because there are so many body-parts missing in the SLEAP file. Is it possible to generate SLEAP predictions where data exist for most body-parts in all of the frames?