Rendered animation different from raw outputs

[edrdos101]

Hi, thanks for sharing your implementation, it helped me immensely when trying to run VideoPose3D on AlphaPose 2D keypoints. There's a bit of an issue that I ran into and was hoping you could help or would have any ideas. When running the render argument, I get a pretty good, albeit glitchy visualisation

However, I also tried plotting 3D keypoints directly - after generating predictions and rebasing height:

    prediction = evaluate(gen, return_predictions=True)

    for subject in dataset.cameras():
        if 'orientation' in dataset.cameras()[subject][args.viz_camera]:
            rot = dataset.cameras()[subject][args.viz_camera]['orientation']
            break

    prediction = camera_to_world(prediction, R=rot, t=0)
    # We don't have the trajectory, but at least we can rebase the height
    prediction[:, :, 2] -= np.min(prediction[:, :, 2])

and when plotting prediction array, here's what I get:

Would you have any ideas what could cause this?

[tobiascz]

Unfortunately, I don't understand your exact problem. In figure 2 you have a 2D and a 3D figure of the predictions array. And you are confused by the appearance of the 2D pose? Or do you think that 2D and 3D do not match?

In case you are wondering about the 2D keypoints. This is the coco keypoint representation. It's used in detectron: keypoints.py.

[edrdos101]

My apologies, I should've been clearer. 2nd figure was showing the 3D output for calling the Temporal Model. Since asking this I realised that I had width/height parameters mixed up for normalisation. It's working fine now