Existing Annotations Do Not Appear in GUI After Pseudo-Labeling or When Continuing Existing Annotations

[nsusic]

Hi, I am having an issue with the behavioral annotations GUI. It appears that when labeling and pseudo labeling that annotations are being saved into targets inserted and when parsing through video frames from the labeled videos, the log states that the labeled behaviors are present. However, in the labeling GUI (both in continuing an existing annotation and the correct labeling after pseudo labeling), the boxes denoting the behaviors are not checked despite behaviors existing in the given frames (both as per the log and targets inserted). I have already verified that the behaviors are labeled and exist in the targets inserted directory, and this is confirmed by the statements within the log. I have also recently updated to the latest version of SIMBA and this has not resolved the issue.

I would appreciate any assistance in solving the issue. Thank you for your help!

Desktop:

• OS: Windows 11
• Python Version: 3.6
• Are you using anaconda? Yes

[sronilsson]

Hi @nsusic! Many thanks for reporting this: I had another user report this a week back, I inserted a potential fix, but I never heard back from the user.. I assumed it was fixed but it appears not. I will see if I can replicate and remove this bug and let you know - I'm travelling for the next couple of days so may be a little slow.

[nsusic]

No problem! I appreciate you taking the time to look into it! If there is anything I can do on my end to help troubleshoot or provide more information, please let me know! Thank you again!

[sronilsson]

@nsusic I think I figured it out...

Recently, I updated the read and write CSV file function in simba to use pyarrow rather than pandas... it speeds things up significantly, but introduced this bug. When reading in the saved annotation CSV, pyarrow interpreted the annotation columns as np.int64 type rather than an int. The checkboxes is looking for a 1 or 0 (if 1, then tick it). As it is neither, it was never checked..

If you upgrade simba with pip install simba-uw-tf-dev --upgrade, how doe sit look on your end?

[nsusic]

Unfortunately, it now appears that the GUI is failing to load entirely. A new window is launched when continuing annotations but the window does not contain anything. Please see the screenshot below. Let me know if there is anything else I can do to help troubleshoot!

[sronilsson]

Thanks for testing @nsusic, do you see any errors in the main OS terminal when this happens?

[nsusic]

Yes; this is the error in the main terminal.

Exception in Tkinter callback Traceback (most recent call last): File "C:\Users\nsusi\anaconda3\envs\SIMBA\lib\tkinter__init.py", line 1705, in call return self.func(*args) File "C:\Users\nsusi\anaconda3\envs\SIMBA\lib\site-packages\simba\SimBA.py", line 320, in continuing=True)) File "C:\Users\nsusi\anaconda3\envs\SIMBA\lib\site-packages\simba\labelling\labelling_interface.py", line 335, in select_labelling_video continuing=continuing) File "C:\Users\nsusi\anaconda3\envs\SIMBA\lib\site-packages\simba\labelling\labelling_interface.py", line 94, in init self.data_df_targets = self.data_df[self.target_lst] File "C:\Users\nsusi\anaconda3\envs\SIMBA\lib\site-packages\pandas\core\frame.py", line 3001, in getitem__ indexer = self.loc._convert_to_indexer(key, axis=1, raise_missing=True) File "C:\Users\nsusi\anaconda3\envs\SIMBA\lib\site-packages\pandas\core\indexing.py", line 1285, in _convert_to_indexer return self._get_listlike_indexer(obj, axis, **kwargs)[1] File "C:\Users\nsusi\anaconda3\envs\SIMBA\lib\site-packages\pandas\core\indexing.py", line 1092, in _get_listlike_indexer keyarr, indexer, o._get_axis_number(axis), raise_missing=raise_missing File "C:\Users\nsusi\anaconda3\envs\SIMBA\lib\site-packages\pandas\core\indexing.py", line 1185, in _validate_read_indexer raise KeyError("{} not in index".format(not_found)) KeyError: "['standing'] not in index"

[sronilsson]

Got it, is there any chance you where trouble shooting changing the project_config.ini, putting the classifier names in lists or similar?

How does this part of the project_config.ini look like on your end?

[sronilsson]

Alternatively! Could it be that you are continuing an annotation for a video, and one of your classifiers is called standing, but, in your project_folder/csv/targets_inserted file representing that video, there isn't a standing column?

[nsusic]

Yes, it is definitely possible. I was attempting to work out of an old directory as the directory in which I was having the issue is not physically accessible at the moment. Unfortunately, I will need a day or two before I can troubleshoot on the PC that was originally having the issue. I hope that is not an issue.

[sronilsson]

Sounds good! I could recreate it that way anyway, and I will insert a better error msg in meantime.

[nsusic]

I just checked and the latest update seems to have fixed the issue! Thank you for your help!

[sronilsson]

Nice one, thanks for letting me know!