Hand Position in Generated OBJ Files

[W-Nabe]

I have successfully generated OBJ files from images using demo.py! Thank you very much.

Currently, I'm aiming to import these OBJ files into Blender for animation purposes.

Here's what I've tried:

1. I extracted JPG files for each frame of an MP4 video.
2. I attempted to recreate the hand movement and position animation in 3D using the sequence of frame images.
3. I imported the sequentially numbered OBJ files into Blender as a Mesh Sequence.

However, the relative hand positions from the image sequence were not reproduced. Instead, only the hand moved around the wrist, as if the wrist were fixed. This wasn't usable for my purpose...

Therefore, I have a question:

Is it possible to obtain OBJ files that contain the relative hand position in each image (or the hand's relative position from the camera in the image) derived from the consecutive video frames input to date_img?

[rolpotamias]

WiLoR is trained to reconstruct 3D hands in the camera space from single image. Therefore it can not provide accurate hand trajectories in a video setting.

However, if the camera of the video is static the reconstructed 3D hands should contain some relative transitions between the frames based on their corresponding translation (cam_t parameter). Please check again if you are saving the OBJ files from this part.

[W-Nabe]

Thank you for your response!

I tried again with a sequence of images extracted from a video. The wrist being the pivot point was due to my modification of the AI to combine multiple OBJs into one.

When I tried again with the original source code, I confirmed that the OBJ files contained positional information, rather than just pivoting around the wrist.

However, I encountered the following issues, as shown in the image:

• Positional instability of the outputted OBJ files: Although the hand positions in the images are almost unchanged, the positions of the outputted OBJ files are unstable and vary greatly.
• Inconsistent numbering of OBJ files: The sequence numbers of the OBJ files are not consistent. This leads to situations where, if there are two hands in an image, their positions might be swapped.

The detection of multiple hands within a single image is perfect! However, it seems difficult to use it for my intended purpose...

If you have any good solutions or suggestions to address these issues, I would be grateful for your advice.