Trouble importing BORIS annotations

[GKNM995]

Hi,

I am trying to import BORIS annotations for a single video with two animals, but I keep getting the error below:

Any idea how I can fix this?

I can send you the csv with the behavior annotations if seeing it would be helpful.

Thank you!

[sronilsson]

Thanks for reporting @GKNM995! To troubleshoot, would you mind sharing the BORIS file, and I can see if I can replicate ?

Also, which version of SimBA are you using (pip show simba-uw-tf-dev) ?

[sronilsson]

PS. If you could also share the SimBA project_config.ini at the same time that would help too

[GKNM995]

Hi,

Thank you for getting back to me so quickly. Here's the info you requested:

Version: 1.95.7 Python 3.10

I have attached the BORIS csv. 2DPI_2573L_2572R.csv

Couldn't attach the config file for some reason, so I have pasted the contents here:

[General settings] project_path = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\project_folder project_name = OF_classifier_training workflow_file_type = csv animal_no = 2 os_system = Windows

[SML settings] model_dir = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models model_path_1 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Body_groom.sav model_path_2 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Face_groom.sav model_path_3 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Locomotion.sav model_path_4 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Rear.sav model_path_5 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Sniff.sav model_path_6 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Stationary.sav model_path_7 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Tail_rattle.sav model_path_8 = C:/Users/gabie/Desktop/OpenField_analysis/SIMBA\OF_classifier_training\models\Turn.sav no_targets = 8 target_name_1 = Stationary target_name_2 = Locomotion target_name_3 = Turn target_name_4 = Sniff target_name_5 = Tail_rattle target_name_6 = Rear target_name_7 = Face_groom target_name_8 = Body_groom

[threshold_settings] threshold_1 = None threshold_2 = None threshold_3 = None threshold_4 = None threshold_5 = None threshold_6 = None threshold_7 = None threshold_8 = None bp_threshold_sklearn = 0.0

[Minimum_bout_lengths] min_bout_1 = None min_bout_2 = None min_bout_3 = None min_bout_4 = None min_bout_5 = None min_bout_6 = None min_bout_7 = None min_bout_8 = None

[Frame settings] distance_mm = 450

[Line plot settings]

[Path plot settings]

[ROI settings]

[Directionality settings]

[process movements]

[create ensemble settings] pose_estimation_body_parts = 16 classifier = None train_test_size = 0.2 under_sample_setting = None under_sample_ratio = None over_sample_setting = None over_sample_ratio = None rf_n_estimators = 2000 rf_min_sample_leaf = 1 rf_max_features = sqrt rf_n_jobs = -1 rf_criterion = entropy generate_rf_model_meta_data_file = None generate_example_decision_tree = None generate_example_decision_tree_fancy = None generate_features_importance_log = None generate_features_importance_bar_graph = None compute_feature_permutation_importance = None generate_sklearn_learning_curves = None generate_precision_recall_curves = None n_feature_importance_bars = None learning_curve_k_splits = None learningcurve_shuffle_data_splits = None

[Multi animal IDs] id_list = Mouse_1,Mouse_2

[Outlier settings] movement_criterion = 1 location_criterion = 1.5 mean_or_median = mean movement_bodypart1_mouse_1 = Nose_1 movement_bodypart2_mouse_1 = Tail_base_1 location_bodypart1_mouse_1 = Nose_1 location_bodypart2_mouse_1 = Tail_base_1 movement_bodypart1_mouse_2 = Nose_2 movement_bodypart2_mouse_2 = Tail_base_2 location_bodypart1_mouse_2 = Nose_2 location_bodypart2_mouse_2 = Tail_base_2

[sronilsson]

Hi @GKNM995 - yes, I can see what is happening.

Your BORIS file is a little different from what SimBA expects. See THIS file as an example for what SimBA expects.

The difference seems to be that in your file, every behavior START and STOP is represented by a single row (which I haven't seen before, but it is great, easy to read, and ensures that every behavior has a start and stop event). However, the BORIS file I link to above (which SimBA expects) has a row for every behavior START event, and a row for every behavior STOP event.

When SimBA finds no START and no STOP in the Behavior type column, it thinks that no behaviors exist in your file (there are only STATE which SimBA doesn't know what to do with).

There are two possible ways of solving it: either (i) is it possible to go back to BORIS and easily save the data in the format that SimBA expects? Or, (ii) if that is not possible, we can write a bit of code that transforms the BORIS filetype you have into the BORIS filestype SimBA expects before feeding it into SimBA?

[GKNM995]

I see. That's helpful. I will get the data in the desired format from BORIS and let you know how it goes (Will report back on Tuesday). Thank you!

[GKNM995]

Hi,

I tried the format you suggested but I get the same error message as before. BORIS csvs attached. 2DPI_LR12PUMP5MG_2572.csv 2DPI_LR12PUMP5MG_2573.csv

[GKNM995]

Hi! I saw your comments on the related issue. Gonna try out those fixes. Thank you!

[GKNM995]

Hi,

I tried the fixes you outlined and got much further than before. However, it is taking a suspiciously long time to append the BORIS data so I think it might be stuck here:

Here's the error message in my command prompt:

Thank you for your help!

[sronilsson]

Thanks for reporting @GKNM995! Let me take a look at this - which version of SimBA are you running when that error happens?

[sronilsson]

I can see one potential issue with the BORIS file:

When SimBA reads in the BORIS annotations in 2DPI_LR12PUMP5MG_2572.csv, it looks to see what FPS is associated with the videos annotated in that file.

Inside 2DPI_LR12PUMP5MG_2572.csv, SimBA finds a single unique video inthe Media file column. Next, it checks what FPS you annotated that video at in the FPS column. However, in this column, SimBA finds something like 10 values separated by ;, like in screengrab below. So the code gets confused, which FPS should it use for the video?

[sronilsson]

And a second issue: I find several NA in the Image index column. THis should be a value representing the frame index when the behavior started/ended, but a bunch of them are missing, do you know why?

I have seen them missing before in the other thread, but that was events that the annotator had forgot to annotate as STOP before the video ended, and BORIS inserted them automatically. In these files however, they seem to be randomly occurring.

[GKNM995]

Those were missing because I forgot to stop a behavior when I started another which I designated as mutually exclusive. I revised the observations in BORIS, and have selected a single FPS for the video. I am using Simba 2.1.7. I updated like you said to do in the other thread 2DPI_LR12PUMP5MG_2572.csv

[sronilsson]

Ah and if you try to import that BORIS data into SimBA, does it still fail?

[GKNM995]

Nope. The new file works!