search

JinluZhang1126 / MixSTE

Official implementation of CVPR 2022 paper(MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video)
193 stars 17 forks source link

Query regarding run.py : inputs_3d[:, :, 0] = 0 #18

Open RHnejad opened 2 years ago

RHnejad commented 2 years ago

Thanks for sharing your interesting work.

Can I ask the reasoning behind inputs_3d[:, :, 0] = 0 ? Considering that it's one of the desired outputs.

Z-Z-J commented 2 years ago

In the process of processing data (run.py line: 114), they remove global offset, but keep trajectory in first position. Because in previous works, such as Videopose3d, these works need to estimate root trajectory (input_3d[:, :, 0]). But in this work, they only estimate root-relative 3D pose, so need input_3d[:, :, 0] = 0

RHnejad commented 2 years ago

Hello and thanks for the answer. Do you also know why they aren't excluding this output completely? Wouldn't it be helpful to estimate the 3D keypoints of 16 joints instead, if this one is always known and expected to be zero?

Z-Z-J commented 2 years ago

Hello and thanks for the answer. Do you also know why they aren't excluding this output completely? Wouldn't it be helpful to estimate the 3D keypoints of 16 joints instead, if this one is always known and expected to be zero?

You can run the training code and find the estimation error of the root joint is close to zero, so it is not necessary to exclude the root joint.