Is there a demo for generating a 3D handpose picture with a 2D hand image

[momo1986]

Hello, @poier ， George.

Thanks for sharing.

I wonder whether this model can be fit to a 2D hand image?

For example, my input is a hand image with normal RGB camera, expected output is a 3D deep hand pose image.

Looks your model is trained on NYU hand dataset.

How can I fit my input data, .e.g, an image to your dataloader?

Is there a potential demo?

Thanks & Regards!

[poier]

Hi,

thanks for your interest.

The models were trained on depth data only and hence will only properly work for depth data. In general, you can however train the method using RGB data. You could also employ the idea of feature mapping, e.g., by learning to map from RGB to depth data and in this way exploit the 3D annotations, which are available for depth data.

I'm not sure if I correctly understand what you want to do with the NYU data. In case you plan to use the RGB data from the NYU dataset to train the model to work on other RGB datasets, I think this is not a good idea. This is because the RGB data from the NYU dataset is significantly distorted and the models trained on this data might not yield quite accurate results on other data.

Hope that helps.

Best regards, Georg