Error in "Import MARS Annotation"

[sara-xinyue-liang]

Describe the bug When I was using simba to import the behaviors labeled in BENTO, the main simba page showed the message "Appending BENTO annotation to video 20240812_crumpling3..." and then it was stuck there. At the same time, some error message showed up in my terminal.

To Reproduce Steps to reproduce the behavior:

1. Go to 'Label behavior' -- "import third-party behavior annotations" -- " import MARS Annotation (select folder with .annot files)"
2. In the directory page, I selected the folder where I have stored the Bento annotation xxx.annot files.
3. Then on the main simba page, I saw the message: "Appending BENTO annotation to video 20240812_crumpling3..."
4. In the terminal of my computer, this is what I saw and an error was indicated in the process of using "import MARS Annotation"

---

(simba) saraliang@dhcp-10-29-170-104 ~ % simba warning||SimBA is not extensively tested beyond python 3.6. You are using python 3.10. If you encounter errors in python>3.6, please report them on GitHub or Gitter (links in the help toolbar) and we will work together to fix the issues! warning||SimBA could not find a FFMPEG installation on computer (as evaluated by "ffmpeg" returning None). SimBA works best with FFMPEG and it is recommended to install it on your computer Exception in thread Thread-1 (run): Traceback (most recent call last): File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/simba/third_party_label_appenders/BENTO_appender.py", line 74, in run annotation_df = pd.read_csv( File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1026, in read_csv return _read(filepath_or_buffer, kwds) File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 626, in _read return parser.read(nrows) File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1923, in read ) = self._engine.read( # type: ignore[attr-defined] File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 239, in read data = self._reader.read(nrows) File "parsers.pyx", line 820, in pandas._libs.parsers.TextReader.read File "parsers.pyx", line 914, in pandas._libs.parsers.TextReader._read_rows File "parsers.pyx", line 891, in pandas._libs.parsers.TextReader._check_tokenize_status File "parsers.pyx", line 2061, in pandas._libs.parsers.raise_parser_error pandas.errors.ParserError: Error tokenizing data. C error: Expected 3 fields in line 5, saw 4

5. Here is my bento annotation file:

   Bento annotation file Movie file(s):

Stimulus name: Annotation start time: 3.333333e-02 Annotation stop time: 1.807000e+02 Annotation framerate: 30.000000

List of channels: fish1

List of annotations: crumpling margin_folding quiescence swimming tacoing

fish1----------

swimming Start Stop Duration 1 73 73 773 1049 277 1678 1927 250 2437 2463 27 2538 2576 39 2596 2605 10 2675 2729 55 3155 3410 256 4341 4521 181 5298 5421 124

crumpling Start Stop Duration 74 377 304 3411 3642 232

margin_folding Start Stop Duration 1404 1677 274 2464 2537 74 2577 2595 19 2606 2674 69 2730 3154 425

quiescence Start Stop Duration 745 772 28 1050 1403 354 1928 2436 509 4013 4340 328 4522 5297 776

tacoing Start Stop Duration 378 744 367 3643 4012 370

---

Screenshots

Desktop (please complete the following information):

• OS: MacOS 14.6.1 (23G93)
• Python Version: 3.10.13 ( I was not able to install 3.6 on Mac M2 when I tried doing that, so I installed simba using python 3.10 instead)
• Are you using anaconda? YES, I created an environment called simba using anaconda.

Thank you!

[sronilsson]

Thanks @sara-xinyue-liang for reporting this, much appreciated. Let's fix this bug.

Sorry the BENTO annotation append code, is a little outdated. I haven't touch it for some years, partly as I don't have any BENTO data at hand no more, so don't know expected input and if it has changed.

Would you be willing to share your annotation file in the original format, perhaps zip it up and drag and drop it into this thread?

And I will make sure Simba can handle it.

[sara-xinyue-liang]

Annotforsimba.zip

Hi @sronilsson , would this work?

Let me know if I can provide anything else that could help!

Sara Liang

[sronilsson]

Thanks @sara-xinyue-liang ! That's all I need for now. I will check this over and get back to you.

[sronilsson]

PS. There is another method for appending BENTO/MARS that was written a little more recently if you use this menu:

It is documented HERE. Not sure if it runs yet on your data but if you have time you can try it and let me know if you see the same error.

[sronilsson]

PS again :)

Don't try the above, it won't work. These files look slightly different from what I have seen in the past.

As I fix the code I want to make sure I don't brake it for people that have other formatted .annot files: Do you know which version of BENTO you used to create these files?

[sara-xinyue-liang]

Thank you for your help! I am using the BENTO I installed in May 2024 from "https://github.com/neuroethology/bentoMAT" I am unable to find what version it is though.

Also, I wonder could you point out the difference between this annot file and the other bento file you have seen in the past? Perhaps sharing an example file? I wonder if I change the file to make them look like the one you have seen whether that will make simba recognize it. I checked the example annot file from https://github.com/sgoldenlab/simba/blob/master/misc/bento_example.annot, but the only difference I noticed is the unit. (and I hope simba can work for both units) but if there is something else that is different, I would like to see if modifying the annot file will make simba recognize it.

Sincerely, Sara Liang

[sara-xinyue-liang]

Quick follow up on that. I quickly tested whether changing unit from frames into seconds can make simba work. Unfortunately I still see the same error:

---

(simba) saraliang@dhcp-10-29-170-104 ~ % simba warning||SimBA is not extensively tested beyond python 3.6. You are using python 3.10. If you encounter errors in python>3.6, please report them on GitHub or Gitter (links in the help toolbar) and we will work together to fix the issues! warning||SimBA could not find a FFMPEG installation on computer (as evaluated by "ffmpeg" returning None). SimBA works best with FFMPEG and it is recommended to install it on your computer Exception in thread Thread-1 (run): Traceback (most recent call last): File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/simba/third_party_label_appenders/BENTO_appender.py", line 74, in run annotation_df = pd.read_csv( File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1026, in read_csv return _read(filepath_or_buffer, kwds) File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 626, in _read return parser.read(nrows) File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1923, in read ) = self._engine.read( # type: ignore[attr-defined] File "/Users/saraliang/anaconda3/envs/simba/lib/python3.10/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 239, in read data = self._reader.read(nrows) File "parsers.pyx", line 820, in pandas._libs.parsers.TextReader.read File "parsers.pyx", line 914, in pandas._libs.parsers.TextReader._read_rows File "parsers.pyx", line 891, in pandas._libs.parsers.TextReader._check_tokenize_status File "parsers.pyx", line 2061, in pandas._libs.parsers.raise_parser_error pandas.errors.ParserError: Error tokenizing data. C error: Expected 3 fields in line 5, saw 4

And I have attached the two annot files I am trying that I have switched the unit from frames to seconds (which still leads to the same error) Annot_in_seconds.zip

[sronilsson]

Yes they were rather subtle the differences:

(i) The file example file on github has a little Ch1---------- delimiter row that marks the split between the meta data and the actual annotation data. I was using that row, thinking it was always there, but that delimiter does not appear in your file so it will fail.

(ii) The file is not utf-8 encoded, it has to be read like an ascii encoded file. The code assumes it is utf-8, and that's why you see the error.

But I have to come back to this and step away from the computer.

One question though: Does BENTO allow you to choose to export the annotation in time or frame format, can they be either?

[sara-xinyue-liang]

Hi @sronilsson ,

"Does BENTO allow you to choose to export the annotation in time or frame format, can they be either?" If I understood your question correctly, BENTO allows the user to choose to export the annotation either in time or frame format. There is a option in"Preference" for that (see screenshot)

[sronilsson]

Thanks @sara-xinyue-liang very helpful I didn't know this, only examples I've seen where is seconds.

What I could do in the code is to check if all the annotation values are exact integers. If they are integers I will assume that the annotations are provided in frame numbers. If they are float values, then I will assume the annotations are provided in seconds. I think this could work.. there may be some issues if you record less than 1 fps like 0.25 fps videos but I will skip those odd cases for now.

EDIT: But please let me know if you can see some issues with that

[sronilsson]

See above, but if you could give it ago with the latest version of SimBA and let me know how it runs?

You can install it with pip install simba-uw-tf-dev --upgrade --no-deps. Should be version 2.0.8.

I think that you might be working fish? Just a heads up, many of the features being calculated out-of-box can be improved for shape shifters like fish, see this thread for an example discussion. I'm happy to help should you want to use your own feature extraction class in simba as documented here, but lets check if the append works first.

[sara-xinyue-liang]

Hi @sronilsson,

I am able to append the label data from Bento using the updated script!

Sara Liang