Add I/O support for MMPose

[niksirbi]

Motivation

Some of our collaborators work with human pose estimation, using MMPose: OpenMMLab Pose Estimation Toolbox and Benchmark.

The would like to import the predicted poses into movement, to leverage our filtering + analysis (and in future, visualisation) features.

Format

The pose predictions are done by an MMPoseInferencer class, the usage and outputs of which are described here. The results are then serialised into json for storage, see code here.

This is an example of how the outputs look like:

{
    "frame_id": 0,
    "instances": [
      {
        "keypoints": [
          [
            961.4871847546324,
            39.44995934640923
          ],
          [
            787.3275313021991,
            255.6230511152969
          ],
        ],
        "keypoint_scores": [
          0.009758014231920242,
          0.005633828230202198,
        ],
        "bbox": [
          [
            496.4014892578125,
            0.298583984375,
            1016.2412109375,
            720.661376953125
          ]
        ],
        "bbox_score": 0.863023042678833
      }
    ]
  },
  {
    "frame_id": 1,
    "instances": [
      {
        "keypoints": [

What we can do

Since movement's data structure already supports keypoints and associated confidence scores, its should be easy to write functions that read/write the "keypoints" and "keypoint_scores" information for all instances and frames from/to such json files.

The public-facing functions can be named load_poses.from_mmpose_file() and save_poses.to_mmpose_file() to conform with our existing naming conventions.

The tricky bit is what to do about the bounding boxes. movement currently doesn't support those (but future support is planned, see https://github.com/neuroinformatics-unit/movement/issues/167). Maybe in the first instance we can only care about the keypoints, and think about the bboxes, at a later stage.

[sfmig]

Maybe in the first instance we can only care about the keypoints, and think about the bboxes, at a later stage.

Potentially we can treat the bbox center as an additional keypoint? 🤔

[niksirbi]

We could, or the two relevant corners (upper left and lower right). Maybe we can designate some reserved keypoint names for such occasions (bbox_start, bbox_end)?

[niksirbi]

This discussion is also relevant for #167

[niksirbi]

@ivanvrlg I've written a "developer guide" specifically for adding new Input/Output functionalities to movement. I hope that it will help clarify the relevant code structure and make it easier to find things. Feel free to comment under the relevant thread on Zulip if things are unclear.

[ivanvrlg]

Thanks Niko, this looks like it will be super helpful! I will post on Zulip if I have any questions.