Spontaneous alternation test

[Monica9577]

Is your feature request related to a problem? Please describe. I'll like to use simba to analyze my spontaneus alternation test in rats. This test consist in letting the animal freely explore a T maze for 10 min and then score how many times the animal does a correct, incorrect or neutral triplet:

If the last and second last-visited arms were diferent and the current choice was diferent from those two arms, then it was considered a correct choice. If the animal chose the one of the already visited arms, the choice was considered incorrect. If the last and second last-visited arms were the same, i.e., the mice consecutively visited the same arm twice, then the current arm choice was considered neutral. The spontaneous alternation rate was calculated as the number of correct choices divided by the total number of correct and incorrect choices.

Describe the solution you'd like I'd like to be posible to score this type of behaviour using ROI in simba. So you draw the ROIs for each arm and from there being able to get a csv with the data of the triplets the animal did. And if it's possible the number of correct, incorrect and neutral ones

Thank you very much in advice!!

[sronilsson]

Good question @Monica9577. Good news is, I actually wrote the code to perform these calculations the other day, documented HERE.

The function gives you, for each video, the percent alternation, total alternation count, same-arm error return errors, alternate error counts, the times when each error occurs, and the times when each alternations occur (together with the specific unique arm sequence that was completed).

The not-so good news is, that the user that requested the function didn’t really get back to me to confirm it was working as expected, and I don’t have hand-annotated data (or commercial software data) to validate it against to confirm I get the same results. As of now, I also don’t have a graphical interface for it, but that would be quick to implement.

To take this forward, would it be possible for you to share a video with me, with some pose-estimation tracking data, for which you know the spontaneous alternation rate? I could confirm it is working, and then insert a menu in the GUI, where you select the arm ROI names from dropdown menus, and click to perform the spontaneous alternation calculations and get the results for each video in a CSV?

[Monica9577]

spontaneus alternation test.zip https://drive.google.com/file/d/1NLr4oHImDFuMYWzwwFFqhaVOcLN44190/view?usp=drive_web Hi ! That's wonderfull ! I'm sending you a folder with an anotatted video and a excel file with the results

Thank you !

El mar, 19 mar 2024 a las 13:36, Simon Nilsson @.***>) escribió:

Good question. Good news is, I actually wrote the code to perform these calculations the other day, documented HERE https://simba-uw-tf-dev.readthedocs.io/en/latest/simba.mixins.html#simba.mixins.feature_extraction_supplement_mixin.FeatureExtractionSupplemental.spontaneous_alternations . image.png (view on web) https://github.com/sgoldenlab/simba/assets/34761092/6607f56a-e811-48ab-bab7-491eb3fe7e6f

The function gives you, for each video, the percent alternation, total alternation count, same-arm error return errors, alternate error counts, the times when each error occurs, and the times when each alternations occur (together with the specific unique arm sequence that was completed).

The not-so good news is, that the user that requested the function didn’t really get back to me to confirm it was working as expected, and I don’t have hand-annotated data (or commercial software data) to validate it against to confirm I get the same results. As of now, I also don’t have a graphical interface for it, but that would be quick to implement.

To take this forward, would it be possible for you to share a video with me, with some pose-estimation tracking data, for which you know the spontaneous alternation rate? I could confirm it is working, and then insert a menu in the GUI, where you select the arm ROI names from dropdown menus, and click to perform the spontaneous alternation calculations and get the results for each video in a CSV?

— Reply to this email directly, view it on GitHub https://github.com/sgoldenlab/simba/issues/349#issuecomment-2007065681, or unsubscribe https://github.com/notifications/unsubscribe-auth/BGGCRMMBC5NIW26KT3MR2EDYZAWOVAVCNFSM6AAAAABE5MXHDOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBXGA3DKNRYGE . You are receiving this because you authored the thread.Message ID: @.***>

[sronilsson]

Cheers I've requested acceess

[Monica9577]

Yes! I have already accepted the access request

[sronilsson]

Thanks - just two questions, which arm is named A, B, and C, i.e., do you use these name code?

And which body-part would you use to score an entry? E.g., nose or center?

[Monica9577]

How you have annotate it in the image is correct,

And for the entries I use the center

[sronilsson]

thanks!

[sronilsson]

@Monica9577 It would be good to get your input on one thing..

You see towards the end in the attached 30s video clip. Your animal is hovering around the entrance to Arm A. SimBA scores the center-point as entering and leaving Arm A a few times in quick succession, while if you score this same sequence manually, you probably would have scored it as a single entry to Arm A rather than several entry and exists.

https://github.com/sgoldenlab/simba/assets/34761092/3d3575a9-2962-48f6-86f4-6c6cfde38967

… I don't know how commercial tools like AnyMaze performs the calculation, but I suspect they count that the entire animal (excluding the tail) have to enter the ROI (or perhaps some threshold percent of the animal body have to the enter the ROI), for an arm entry to count.

One potential solution is that we also do this, e.g., if you say 50% in a drop-down, it would count as an entry while half or more of the animal is in an arm (like the left example below), or if you say 100%, it would count as an entry only if entire animal is an arm (like the right example below).

However, I have never used AnyMaze and I have never scored spontaneous alternation, manually, what is the typical procedure and criterion for scoring an arm entry?

[Monica9577]

Hi !

When I use AnyMaze to score this type of tests, I usually put that at least 80% of the animal have to enter the ROI in order to count as an entry (or an exit). So if it's possible to score it that way, it would be perfect!

Thank you so much for the effort and the time !

[sronilsson]

Ah that's cool thanks that's good to know - then I will reach out if I have any AnyMaze questions :)

... I noticed AnyMaze have this visualization below, and we will probably need something similar, which also shows the inset names of the three most recent visit arms. So users can visually confirm that the numbers are accurate to make it useful. I need some time to write it and I will get back to you.

[Monica9577]

That's awesome !

Yes feel free to ask any AnyMaze question, no problem !

[sronilsson]

Alright, one AnyMaze question!

Check the attached video. Around the 30s-35s mark, the animal is nearly disappearing out of the image, and therefore out of the ROI. I see the AnyMaze example videos have a little safety distance between the edge of the video frame and the end of the arms, but we don't have that in this case. Do you know how AnyMaze solve this issues?

https://github.com/sgoldenlab/simba/assets/34761092/108e4001-7b04-4192-aed0-ed6961fb74b4

[sronilsson]

... could it be that in AnyMaze, you have to cover the entire maze with ROIs. If that is the case, and the animal "disappears", the animal is placed in the ROI where it was last reliably detected for the frames where it is missing?

[Monica9577]

In AnyMaze I had the same problem The best way I found to solve it is telling the program to only score it as an exit if the animal enters another ROI, and and we establish an ROI at the intersection of the three arms, called the center. In this way it would only count as an exit from arm A if the animal goes through the center.

[sronilsson]

Makes sense! Let's do that

[sronilsson]

@Monica9577 A couple more question about how spontaneous alternation is calculated so we get it right, thanks for help so far .

1) Can you check the following video and let me know if the calculations printed on the side are accurate or not? It is calculated as 100% of the animal has to enter / exit an arm for it to count.

https://drive.google.com/file/d/1g9AYspK56d-de_mf5Og9W8WNwzjpGCwt/view?usp=sharing

2) In this second example below, it is calculated as 60% of the animal that has to enter / exit an arm for it to count - and we get some "revisit" errors where e.g., the animal visits say A->A->A or B->A->A. Next, the animal visits C, making the sequence A->A->C. The visit to arm C, should that be counted as an error? I guess I am having a hard time getting me head around it as the animal has to visit either C or B in order to get the chance to fulfill a successful sequence in the next arm visit, but I'm guessing it should be an error.

https://drive.google.com/file/d/1wtRq-HGl0Mralik24-n2GXk4iY3Rqece/view?usp=sharing

Thanks!

[Monica9577]

Personally, I believe that when 100% of the animal is counted, there are several times when it would manually qualify as an entry, but in this example it is not counted as such.

On the other hand, when 60% of the animal is counted it is possible to have re-entry errors.

I think this model looks very good. Maybe a solution is to set a value between 100 and 60, such as 80% to qualify as an entry to the arm.

[sronilsson]

Thanks @Monica9577 I was thinking more about the actual spontaneous alternation protocol:

If an animal has visited arm A followed by the center, followed by arm A, followed by the center, and followed by arm A, the current three-arm sequence would read A A A. If the animal next visits C, is that visit to arm C coded as an error?

[Monica9577]

Yes if he visits A- A- C, it would be an error

[sronilsson]

Thanks @Monica9577 !

So you would have a menu like this (haven’t documented anything as yet so type it here):

In ARM DEFINITIONS you would just select which ROI that you have drawn represent which arms.

In ANIMAL SETTINGS menu you would set:

POSE ESTIMATION THRESHOLD: Any body-part detected with lower probability than this will be omitted when computing the animal shape.

ANIMAL AREA (%): The percent of the animal the have to enter/exit a region for it to count.

ANIMAL BUFFER (MM): Sometimes you may want to increase the animal size with a little safety buffer in millimeter (see THIS for example of what I mean)

SAVE DETAILED DATA: If True, give you a separate CSV for each video, with the different arm sequences and successful alternations and errors with the frame numbers when they happened. Otherwise you will just get a summary file with the stats for each video containing alternation rate, error counts, alternation count, return errors and alternation errors.

VERBOSE: Useful for troubleshooting, if True, print out what processing is being done, otherwise you will just get a print out when each file is complete.

Do you think we are missing anything?

I have only really tested it so it runs as expected on F1_HAB video you shared, so may be some errors that could arise that I haven't been able to anticipate. If you have more videos I could test it on those before sharing the code.

[Monica9577]

I am still testing this function and comparing it with manual analysis. So far it seems very good, it would be great if a list with the order in which the animal has entered the different arms could be included in the csv sheet of results. (Ex. A,B,B,B,C,A,B,C,A....)

[sronilsson]

@Monica9577 - definitly - thanks. How about if you tick the detailed data checkbox, you get an additional CSV for each video file looking something like this? Would that work? (with one of these files per video)

[sronilsson]

I inserted this output in version 1.90.8, if you set the SAVED DETAILED DATA dropdown to True, but please let me know how goes.

[sronilsson]

FYI I realized you probably want the entry / exit times as well listed alongside, in frames and time, so added it, you should expects something line this.

F1 HAB_arm_entry_sequence.csv