deepof_11 labeling scheme

[olivamb]

Hi,

Thank you for providing this great software tool. We want to analyze the social behavior between two mice and applied multi animal DLC with 11 body parts. They correspond to the 14 body parts you used without the three on the tail (Tail_1, Tail_2, Tail_tip). Since you mentioned that you no longer use these three body parts in DeepOF, we thought we could create a project based on the default labeling scheme and simply omit the line exclude_bodyparts=["Tail_1", "Tail_2", "Tail_tip"],. Unfortunately, we run into problems while the project is being created. First it says: All-NaN slice encountered. The calculation of distances seems to work. But when calculating angles, it stops and says: "['T_Tail_1'] not in index". Is there a way to use the DLC data with only 11 body parts without defining a custom labeling scheme? Or could you suggest a way to set up the custom scheme to best match the original deepof_14 definition?

Thank you very much for your support! Kind regards, Oliver

Setting up project directories...
Loading trajectories...
Smoothing trajectories...
Interpolating outliers...
Iterative imputation of ocluded bodyparts...
Detecting arena...

All-NaN slice encountered

Computing distances...
Computing angles...

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/deepof/data.py:567, in Project.get_angles(self, tab_dict, verbose)
    564 for clique in bridges:
    565     dat = pd.DataFrame(
    566         deepof.utils.angle(
--> 567             np.array(tab[clique]).reshape([3, tab.shape[0], 2])
    568         ).T
    569     )
    571     dat.columns = [tuple(clique)]

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/pandas/core/frame.py:3813, in DataFrame.__getitem__(self, key)
   3812         key = list(key)
-> 3813     indexer = self.columns._get_indexer_strict(key, "columns")[1]
   3815 # take() does not accept boolean indexers

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/pandas/core/indexes/multi.py:2623, in MultiIndex._get_indexer_strict(self, key, axis_name)
   2621 indexer = self._get_indexer_level_0(keyarr)
-> 2623 self._raise_if_missing(key, indexer, axis_name)
   2624 return self[indexer], indexer

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/pandas/core/indexes/multi.py:2641, in MultiIndex._raise_if_missing(self, key, indexer, axis_name)
   2640 if cmask.any():
-> 2641     raise KeyError(f"{keyarr[cmask]} not in index")
   2642 # We get here when levels still contain values which are not
   2643 # actually in Index anymore

KeyError: "['T_Tail_1'] not in index"

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
Cell In[4], line 1
----> 1 my_deepof_project = my_deepof_project_raw.create(force=True)

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/deepof/data.py:700, in Project.create(self, verbose, force, debug, test, _to_extend)
    697     distances = self.get_distances(tables, verbose)
    699 if self.angles:
--> 700     angles = self.get_angles(tables, verbose)
    702 if self.areas:
    703     areas = self.get_areas(tables, verbose)

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/deepof/data.py:578, in Project.get_angles(self, tab_dict, verbose)
    576         angle_dict[key] = dats
    577 except KeyError:
--> 578     raise KeyError(
    579         "Are you using a custom labelling scheme? Out tutorials may help! "
    580         "In case you're not, are there multiple animals in your single-animal DLC video? Make sure to set the "
    581         "animal_ids parameter in deepof.data.Project"
    582     )
    584 # Restore original index
    585 for key in angle_dict.keys():

KeyError: "Are you using a custom labelling scheme? Out tutorials may help! In case you're not, are there multiple animals in your single-animal DLC video? Make sure to set the animal_ids parameter in deepof.data.Project"

[joeribordes]

Hi Oliver, cool to see that you are working with DeepOF! If I am not mistaken, you would still need to add the line "exclude_bodyparts=["Tail_1", "Tail_2", "Tail_tip"]" otherwise DeepOF is trying to search for these bodyparts but can't find it. Let me know if that solves the problem! Best , Joeri

[lucasmiranda42]

Hi, Oliver! Joeri is right. Feel free to post your current code if you need further assistance!

[olivamb]

Hi Joeri and Lucas, thanks for your ideas. We tried that and unfortunately it does not work, either. The software is missing these bodyparts when running the command to exclude them KeyError: "labels ['T_Tail_1' 'T_Tail_2' 'T_Tail_tip' 'I_Tail_1' 'I_Tail_2' 'I_Tail_tip'] not found in level" Should we use a custom labeling scheme or add empty columns with the missing titles to the csv file? Best, Oliver

We just adopted the code from the tutorial:

my_deepof_project_raw = deepof.data.Project(
                project_path=os.path.join("DeepOF/SIT"),
                video_path=os.path.join("DeepOF/SIT/Videos/"),
                table_path=os.path.join("DeepOF/SIT/Tables/"),
                project_name="deepof_SRmice_project",
                arena="circular-autodetect",
                animal_ids=["T", "I"],
                table_format="csv",
                video_format=".mp4",
                exclude_bodyparts=["Tail_1", "Tail_2", "Tail_tip"],
                video_scale=600,
                smooth_alpha=1,
                exp_conditions=None,
)

my_deepof_project = my_deepof_project_raw.create(force=True)

The complete output is as follows:

Setting up project directories...
Loading trajectories...
Smoothing trajectories...

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[4], line 1
----> 1 my_deepof_project = my_deepof_project_raw.create(force=True)

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/deepof/data.py:681, in Project.create(self, verbose, force, debug, test, _to_extend)
    671     self.set_up_project_directory(debug=debug)
    673 self.frame_rate = int(
    674     np.round(
    675         pims.ImageIOReader(
   (...)
    678     )
    679 )
--> 681 tables, quality = self.load_tables(verbose)
    682 if self.exp_conditions is not None:
    683     assert (
    684         tables.keys() == self.exp_conditions.keys()
    685     ), "experimental IDs in exp_conditions do not match"

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/deepof/data.py:459, in Project.load_tables(self, verbose)
    456 if self.exclude_bodyparts != tuple([""]):
    458     for k, tab in tab_dict.items():
--> 459         temp = tab.drop(self.exclude_bodyparts, axis=1, level="bodyparts")
    460         temp.sort_index(axis=1, inplace=True)
    461         temp.columns = pd.MultiIndex.from_product(
    462             [sorted(list(set([i[j] for i in temp.columns]))) for j in range(2)]
    463         )

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/pandas/util/_decorators.py:331, in deprecate_nonkeyword_arguments.<locals>.decorate.<locals>.wrapper(*args, **kwargs)
    325 if len(args) > num_allow_args:
    326     warnings.warn(
    327         msg.format(arguments=_format_argument_list(allow_args)),
    328         FutureWarning,
    329         stacklevel=find_stack_level(),
    330     )
--> 331 return func(*args, **kwargs)

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/pandas/core/frame.py:5399, in DataFrame.drop(self, labels, axis, index, columns, level, inplace, errors)
   5251 @deprecate_nonkeyword_arguments(version=None, allowed_args=["self", "labels"])
   5252 def drop(  # type: ignore[override]
   5253     self,
   (...)
   5260     errors: IgnoreRaise = "raise",
   5261 ) -> DataFrame | None:
   5262     """
   5263     Drop specified labels from rows or columns.
   5264 
   (...)
   5397             weight  1.0     0.8
   5398     """
-> 5399     return super().drop(
   5400         labels=labels,
   5401         axis=axis,
   5402         index=index,
   5403         columns=columns,
   5404         level=level,
   5405         inplace=inplace,
   5406         errors=errors,
   5407     )

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/pandas/util/_decorators.py:331, in deprecate_nonkeyword_arguments.<locals>.decorate.<locals>.wrapper(*args, **kwargs)
    325 if len(args) > num_allow_args:
    326     warnings.warn(
    327         msg.format(arguments=_format_argument_list(allow_args)),
    328         FutureWarning,
    329         stacklevel=find_stack_level(),
    330     )
--> 331 return func(*args, **kwargs)

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/pandas/core/generic.py:4505, in NDFrame.drop(self, labels, axis, index, columns, level, inplace, errors)
   4503 for axis, labels in axes.items():
   4504     if labels is not None:
-> 4505         obj = obj._drop_axis(labels, axis, level=level, errors=errors)
   4507 if inplace:
   4508     self._update_inplace(obj)

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/pandas/core/generic.py:4544, in NDFrame._drop_axis(self, labels, axis, level, errors, only_slice)
   4542     if not isinstance(axis, MultiIndex):
   4543         raise AssertionError("axis must be a MultiIndex")
-> 4544     new_axis = axis.drop(labels, level=level, errors=errors)
   4545 else:
   4546     new_axis = axis.drop(labels, errors=errors)

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/pandas/core/indexes/multi.py:2259, in MultiIndex.drop(self, codes, level, errors)
   2244 """
   2245 Make new MultiIndex with passed list of codes deleted
   2246 
   (...)
   2256 dropped : MultiIndex
   2257 """
   2258 if level is not None:
-> 2259     return self._drop_from_level(codes, level, errors)
   2261 if not isinstance(codes, (np.ndarray, Index)):
   2262     try:

File ~/anaconda3/envs/deepof/lib/python3.9/site-packages/pandas/core/indexes/multi.py:2311, in MultiIndex._drop_from_level(self, codes, level, errors)
   2309 not_found = codes[values == -2]
   2310 if len(not_found) != 0 and errors != "ignore":
-> 2311     raise KeyError(f"labels {not_found} not found in level")
   2312 mask = ~algos.isin(self.codes[i], values)
   2314 return self[mask]

KeyError: "labels ['T_Tail_1' 'T_Tail_2' 'T_Tail_tip' 'I_Tail_1' 'I_Tail_2' 'I_Tail_tip'] not found in level"

[joeribordes]

Hi Oliver, Hmm yeah strange. It might be that there is a DeepOF bug when not using the 14 labels, even though it should work without them, @lucasmiranda42 what do you think?

Oliver, do you know if you are running the latest version of DeepOF? And would you mind sending us one example video and CSV file to see if we can recreate the issue? You can send it to joeri.bordes@inserm.fr

[lucasmiranda42]

Dear Oliver,

Apologies for the mistake and slight delay. I believe adopting a custom labelling scheme would indeed be the way to go, then. I think this should be unnecessary in future versions, however. Specifying parts to ignore should remove them from the underlying graph that DeepOF checks when raising the error you see. I'll mark this as a good first issue for contributing, and take action as soon as we can!

For now, you should define a custom graph as depicted in this tutorial. Below how the code would look:

deepof_custom = { # Note that this is the same original graph without the tail nodes
    "Nose": ["Left_ear", "Right_ear"],
    "Spine_1": ["Center", "Left_ear", "Right_ear"],
    "Center": ["Left_fhip", "Right_fhip", "Spine_2"],
    "Spine_2": ["Left_bhip", "Right_bhip", "Tail_base"],
}

my_deepof_project_raw = deepof.data.Project(
                project_path=os.path.join("DeepOF/SIT"),
                video_path=os.path.join("DeepOF/SIT/Videos/"),
                table_path=os.path.join("DeepOF/SIT/Tables/"),
                project_name="deepof_SRmice_project",
                arena="circular-autodetect",
                animal_ids=["T", "I"],
                table_format="csv",
                video_format=".mp4",
                # exclude_bodyparts=["Tail_1", "Tail_2", "Tail_tip"], # No need to exclude these anymore
                bodypart_graph=deepof_custom, # Pass the graph defined above
                video_scale=600,
                smooth_alpha=1,
                exp_conditions=None,
)

Please let us know how it goes, and feel free to comment if you still have issues!

Best, Lucas

[olivamb]

Dear Joeri and Lucas, Thank you a lot for your assistance! The custom labeling scheme worked without problems. If the 11-point labeling would be supported in a future version, it would be nice, but for now I see the problem as solved. Best, Oliver

[lucasmiranda42]

Dear Oliver,

Glad to hear! I'll leave the issue open for now, and close it as soon as we have time to work on the suggested enhancement.

Best! Lucas

[NoCreativeIdeaForGoodUserName]

"deepof_11" (i.e. all body parts as in deepof_14 but without tail) is now available as an option during project creation (starting with version 0.7 of deepof). I close this issue now.