OpenPose detection with upside-down cameras + OpenPose model choice # Error with scaling on static pose

[reubenlindroos]

This issue arrising when following the standard pipeline for scaling using an A-pose.

[davidpagnon]

Thank you for posting here what we shared per email! I used your data, and as you will see, your issues don't have much to do with scaling.

It gives me the occasion to tackle two potential issues that other users may come accross:

• Camera orientation during capture,
• OpenPose model choice.

Camera orientation

According to your Calib.toml file, cameras 4, 8, and 9 are upside down. OpenPose is really bad with estimating picture from upside-down position. Four ways to fix it:

• Turn your cameras prior to capture.
• Rotate your video with ffmpeg, and then change your extrinsic parameters in your Calib.toml file. I can provide you guidance for this, probably with a ready-to-use script if I take the time for it.
• Use the OpenPose frame_rotate flag. It should be easier, but I have never tried.
• Just delete the cameras in the Calib.toml file, and the corresponding camera subfolders in the pose-2d folder (idem in pose-2d-tracked if applicable). For now, this is the solution I choose, even if it means using only 6 cameras instead of the 9 available.

By the way, if you want to see how bad OpenPose performs on flipped cameras, try the Utilities/json_display_without_img.py script; and for the visualization in Maya, see there.

OpenPose model

In the pose_model section of your Config.toml file, you ask for using the body_25b skeleton. However, from the inspection of your json files, I can tell that ran OpenPose with the body_25 one. The latter gets rid of the (artificially created) midhip point. Also, keypoint order is different, as you can see on the image above (from this article).

More specifically, in the 2d-tracking part of your Config.toml file, you specified that you would track the neck for getting rid of other detected people. The ID of the neck point is 17 in the body_25b model, which corresponds to the right ear in the body_25 one. And the ear is less reliably detected, hence some additional issues in the tracking step, which are pased on to all other steps. OpenSim scaling and inverse kinematics may not be working as expected, too.

The solution to this is just to switch BODY_25B to BODY_25 in the Config.toml file (or to run OpenPose anew with the body_25b model .

With this correction, I just ran the tracking, the triangulation, and the filtering steps without changing any other parameter (with only 6 cameras, though), and results are convincing: wrist and ankle points don't go all over the place as they did before, and OpenSim scaling and inverse kinematics run as expected!

Subsidiary point

For future reference, starting from v0.2.3, frames_range has been changed to frame_range, and openpose_model to pose_model. This could explain TypeError: 'NoneType' object is not subscriptable or TypeError: eval() arg 1 must be a string, bytes or code object