Kinetic Energy Decomposition

[niksirbi]

This feature was suggested by @dimokaramanlis, who finds it useful in analysing his pose tracking data.

Definition

If we consider each individual's set of keypoints (pose) as a classical system of bodies in physics (with unit masses for simplicity), we could compute the total kinetic energy of that system as:

$$ K{total} = \sum{i} \frac{1}{2} || \mathbf{v}_i(t) ||^2 $$

Where $\mathbf{v}_i(t) = [v_i^x(t), v_i^y(t)]$ is the 2D (x,y) velocity vector of the $i_{th}$ keypoint at time $t$.

We can decompose $K_{total}$ into two components:

1. The translational kinetic energy $K_{trans}$ which is associated with the motion of the centre-of-mass. We can get this if we first compute the centre-of-mass velocity $\mathbf{V}(t)$ (across $M$ keypoints).

$$ \mathbf{V}(t) = \frac{1}{M} \sum_i \mathbf{v}_i(t) $$

$$ K_\text{trans} = \frac{1}{2} ||\mathbf{V}(t)||^2 $$

1. The rotational kinetic energy $K_{rot}$ which is associated with the motion relative to the centre-of-mass. We can derive this simply as the remaining kinetic energy:

$$ K{rot} = K{total} - K_{trans} $$

Applications

• Dimos has found this to be extremely useful in identifying action onsets, e.g. a mouse initiating a turn and run. In such cases, there is a rapid rise in kinetic energy, with $K_{rot}$ often preceding $K_{trans}$.
  • He also uses this to proofread and interpret behavioural syllables output by keypoint-moseq, where each syllable could have characteristic energy signatures (e.g. some are purely rotational).

Implementation

This should be doable in a few lines of xarray code, with our existing dataset structure. The output could be a DataArray with dimensions time, individuals, and energy, where energy can have two coordinates - translational and rotational.

It will be facilitated by implementing a norm() function in utils/vector.py, which @sfmig was planning to do anyway.

Considerations

• For this to work, one needs at least 2 keypoints per individual, and more than that would be desirable for a meaningful result.
• In theory, if one has multiple individuals behaving like a swarm/flock, the same decompositions could be done across individuals instead of across keypoints (viewing the whole swarm as a system).

[niksirbi]

Two additional considerations brought up by Dimos:

• In some cases it may make sense to exclude some keypoints from the computation. An example of this is the tail tip in mice, which is often unreliably detected, and can have big "passive" movements due to inertia. Ultimately, we should leave this choice to the user, but may be worth adding a note about it in the docstring.
• Sometimes the (relative) masses of different body parts are known, and it may make sense to include them in the computation (the formulas above assume unit masses). We could provide an optional argument that has the form of a dictionary {keypoint_id: weight}.

[sfmig]

I like this idea! And the suggestion of computing these at the individual level or at the collective level (for a flock) could be cool.

But maybe there are metrics that are more informative for a collective. From my experience with flocking behaviour, common metrics relate to how aligned are the velocity vectors of the individuals, e.g. with the average velocity of the flock or with their neighbours' velocity vectors (see Fig1 here for a metric along these lines). Note that with kinetic energy you loose the alignment info. Maybe we could open a new issue on implementing metrics for flocks - what do you think?

I think the kinetic energy computation per individual is not currently a very demanded feature, but as you say should be easy to implement, and seems useful for analysis. Maybe a good one to get users to try out and report back?

[niksirbi]

Yeah I think metrics for flocking behaviour are interesting in their own right, but that should be a separate issue.

For this one, let's focus on computing the kinetic energy per individual only. I agree it's not a high priority, but should be very quick as it's a very well-defined task (famous last words). It would be nice to have an additional metric in the kinematics module.