Open simon3dv opened 8 months ago
Hi @simon3dv,
Besides, we update the frame-level textual descriptions for each whole-body pose. Please download it here and refer to this usage guidance PoseTEXT_README.
Thanks! By the way, I am using PoseScript to generate captions for some open-dataset(3DPW, DNA-Rendering, ...).
The PoseScript auto captioning pipeline normalize SMPL to orient_y=0(looking at camera) before captioning. It leads to some problems when extending to captioning image with invisible body part.
I tried using keypoint confidence from RTMPose to detect which part is invisible, but found the confidence does help but is not robust enough to represent real visibility. I also tried to use the inconsistent results from RTMPose and SSLPose as invisibility, but the results are bad yet. Here are some examples.
(a)Left hand is occluded, right hand is visible, but incorrectly detect right elbow to be occluded.
(b)Left hand is not occluded, but detect to be occluded.
Do you have solution to fix the caption in these two cases? Did you also use confidence from pose detection to judge which part is invisible ?
Hi authors, I am very amazed by your work. I notice you use face recognition, posescript and handscript to generate 15.6M frame-level descriptions and have some questions about it.