convert agora data and visual smpl is wrong?

open-mmlab / mmhuman3d

OpenMMLab 3D Human Parametric Model Toolbox and Benchmark

https://mmhuman3d.readthedocs.io/

Apache License 2.0

1.24k stars 137 forks source link

convert agora data and visual smpl is wrong? #124

Open Gzzgz opened 2 years ago

Gzzgz commented 2 years ago

I convert agora data , the result is like this: and I found ori data has 'Cam' folder and you not used it.

Gzzgz commented 2 years ago

the same as surreal dataset

Gzzgz commented 2 years ago

Maybe surreal datdaset has some errors.

pangyyyyy commented 2 years ago

@Gzzgz Thanks for raising the issue!

For surreal, I found that some of the ground-truth were noisy/ wrong when I visualised them. For the issue you raised here (https://github.com/open-mmlab/mmhuman3d/issues/121), there is a problem of left-right of flipping 3d keypoints and smpl poses that has been confirmed by the author, which I will push up the fix soon. I also realised that the global orientation is wrong for some samples but I have yet to obtain a reply from the author for this.

For agora, I will take a look at it soon.

Gzzgz commented 2 years ago

@pangyyyyy OK, THX.

Gzzgz commented 2 years ago

？

pangyyyyy commented 2 years ago

Hi @Gzzgz,I have pushed up the fix for AGORA's global orientation in PR #143. I've made some visualisations and the smpl poses looks correct now.

frame_000005(1) frame_000008(1) frame_000001(1)

For SURREAL, I have added the left-right flipping of 3d keypoints and smpl poses - a problem for the issue you raised here (#121) that has been confirmed by the author. However, I have yet to obtain any reply from SURREAL's authors regarding the incorrect global orientation.

YongtaoGe commented 2 years ago

Hi, @pangyyyyy Could you give me an example visualization script for the converted agora dataset? Thanks!

caizhongang commented 2 years ago

Hi, @pangyyyyy Could you give me an example visualization script for the converted agora dataset? Thanks!

We will have add a data visualizer tool for HumanData format soon ;)

Gzzgz commented 2 years ago

OK ， But I maybe found it has some errors . in training processing, the kpt3d-loss is bigger than not use it. And I vis just look like this:[some images are ok]

And I checkout your codes that like agora dataset codes. I maybe think the agora-dataset code is wrong.

Gzzgz commented 2 years ago

I think this code should be like :

But I found realy it has a little angle error in some images.like it:

Gzzgz commented 2 years ago

@caizhongang can you doing it with SMPL or SMPLX in cam?

pangyyyyy commented 2 years ago

Hello @Gzzgz, thanks for your inputs, the keypoints3d problem and incorrect global orientation problem is fixed in #202.

I've visualised more images of the converted annotations but they look alright to me. Do you mind providing an example of the annotation id for the problematic visualisations?

John-Yao commented 1 year ago

@Gzzgz @caizhongang
Could you give me an example visualization script for the converted agora dataset? Thanks! The official visualization tools is too complex to visual single image results