search

una-dinosauria / 3d-pose-baseline

A simple baseline for 3d human pose estimation in tensorflow. Presented at ICCV 17.
MIT License
1.42k stars 356 forks source link

Question about visualizing 3d pose output on MPII #23

Closed RichardKov closed 6 years ago

RichardKov commented 7 years ago

Hi Julieta,

Your work of 3d pose is very awesome! I am trying to run your work on MPII dataset to get some 3d pose results. However, the output looks very strange. I think that there may be some problem with the 2d detections' output. The resolution of Human3.6m is 1000x1000 and the height of the person is around 1/3 of the image. But the data in MPII dataset vary a lot in both image size and human size. How do you process the 2d pose results of MPII dataset to get good performance? Hope to get your help!

una-dinosauria commented 7 years ago

Hi @RichardKov,

the output looks very strange. I think that there may be some problem with the 2d detections' output. The resolution of Human3.6m is 1000x1000 and the height of the person is around 1/3 of the image. But the data in MPII dataset vary a lot in both image size and human size.

Our network is definitely using information in the size of 2d pixels to estimate 3d sizes but, as you note, this is biased towards the image and people sizes in H3.6M.

How do you process the 2d pose results of MPII dataset to get good performance?

We did not pre-process the MPII images in any way. Our network still manages to predict 3d poses that seem realistic, as long as you ignore the "absolute" 3d size of the person.

Estimating actual sizes of people in 3d from arbitrary 2d images would probably require more holistic image reasoning, including plane estimation and depth estimation, and a varied dataset of images in the wild with such data, which is hard to get. I'd say that this is an interesting open research question.