Add compatibility with newer BORIS annotation format

[florianduclot]

In recent BORIS versions (v8+?), the export of behavioral events in a tabular format used as annotations files in SimBA has changed. This commit thus enables SimBA to import these annotations files in a new format, while maintaining compatibility with the now-older format. I'm uncertain as to when exactly the format changed, but exporting the list of events as a tabular file in BORIS now yields a nice data frame without having to skip lines:

Observation id	Observation date	Description	Observation duration	Observation type	Source	Media duration (s)	FPS	Subject	Behavior	Behavioral category	Behavior type	Time	Media file name	Image index	Image file path	Comment
testobs_new	2023-05-09 17:15:54		238.739	Media file(s)	player #⁠1:videos/testExp/Video11.mp4	315.7	59.94		behavior1		START	5.355	videos/testExp/Video11.mp4	NA	NA
testobs_new	2023-05-09 17:15:54		238.739	Media file(s)	player #⁠1:videos/testExp/Video11.mp4	315.7	59.94		behavior1		STOP	9.626	videos/testExp/Video11.mp4	NA	NA
testobs_new	2023-05-09 17:15:54		238.739	Media file(s)	player #⁠1:videos/testExp/Video11.mp4	315.7	59.94		behavior2		START	11.178	videos/testExp/Video11.mp4	NA	NA
testobs_new	2023-05-09 17:15:54		238.739	Media file(s)	player #⁠1:videos/testExp/Video11.mp4	315.7	59.94		behavior2		STOP	11.728	videos/testExp/Video11.mp4	NA	NA

Unfortunately, this means SimBA cannot directly import these newly-formatted files from BORIS (not without manually editing them, of course). This pull request simply enables SimBA to do just that.

This is done by adding a quick version checker seeking for the presence of a column name found only in the new format, followed by the adjustment of expected columns headers to match what SimBA is already expecting. This way, importing a BORIS annotation file that uses the now-older format should still work.

@sronilsson, I have tested using both importing ways:

• using the "Import BORIS Annotation (select folder with .csv files)" button. This does give a csv file with the behaviors appended as the last columns in project_folder/csv/targets_inserted. The log in the main SimBA window is:

  Processing BORIS for 1 file(s)...
  Found 3 annotated behaviors in 1 files with /home/florian/PycharmProjects/Simba/external_data/exports/mixed_observations directory
  The following behavior annotations where detected in the boris directory:
  behavior1
  behavior2
  behavior3
  Appending BORIS annotations to Video11 ...
  Saved BORIS annotations for video Video11...
  SIMBA COMPLETE: BORIS annotations appended to dataset and saved in project_folder/csv/targets_inserted directory (elapsed time: 1.1996s)

• using the (newer?) "Append third-party annotations" window. This also gives a csv file with the behaviors appended as the last columns in project_folder/csv/targets_inserted. The log in the main SimBA window is:

  Processing 1 BORIS file(s)...
  Reading in 1 BORIS annotation files...
  Processing annotations for Video11 video...
  Saved BORIS annotations for video Video11...
  SIMBA COMPLETE: BORIS annotations appended to dataset and saved in project_folder/csv/targets_inserted directory (elapsed time: 1.3718s)

I have not tested using an older BORIS format file as I do not have one with matching video and pose tracking files.

Hope this helps,

[florianduclot]

Sorry for the terrible display of the table here... The horizontal scrolling seems to work just fine with editing the message and looking at the "Preview" tab. Here's a screenshot to circumvent this oddity:

[florianduclot]

A nit: could we make _is_new_boris_version a standalone function in third_party_label_appenders.tools, so a single function called by both the old and the new BORIS appender methods?

Glad you bring this up... I wondered about that and forgot to mention it in the PR. I was a bit confused as to whether or not one of the ways to import annotations would be deprecated so I was unsure about using one to refer a function from the other. I'll include your request then!

[florianduclot]

Done: 7e3d3ac

I tested again the same way as described above and it still gives the same results and log outputs.

[sronilsson]

Hi @florianduclot! You don't happen to have a very small CSV example of the new BORIS format? Something that we can stick in the repo I can link to in the docs as expected input?

[florianduclot]

Hi @sronilsson, Here's the file I used to test that branch (it's less than 20 Kb but it can of course be cut down further if preferred): boris_new_example.csv

[sronilsson]

Thanks!