search

facebookresearch / InterHand2.6M

Official PyTorch implementation of "InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image", ECCV 2020
Other
676 stars 92 forks source link

question about color type of hands in dataset #78

Open MengHao666 opened 2 years ago

MengHao666 commented 2 years ago

there are a lot of colors of hands in your dataset, some of them are purely gray image, and other of them are rgb image with colors

mks0601 commented 2 years ago

What is your question?

rohitdavas commented 2 years ago

Hi, The interhand dataset has mostly yellowish images or whitish hands. The hands do not look quite real as in natural light sense. After analysis I found that hand images color in R and G space and B space is quite close. And hence we can use color stretching to come to real looking images. Is there any better way ?

(64, 64, 4)

before.

R values. (min, max) = (0.0, 146.0) G values. (min, max) = (0.0, 139.0) B values. (min, max) = (0.0, 133.0)

after stretch. R values. (min, max) = (0.0, 225.2) stretch. G values. (min, max) = (0.0, 175.10000000000002) stretch. B values. (min, max) = (0.0, 156.4)

Figure_3

mks0601 commented 2 years ago

@rohitdavas That's true. The unrealistic color of images is due to the capture system's light setting and cameras' color calibration.

rohitdavas commented 2 years ago

So is there any better way to make them look realistic ? Otherwise the dataset will not be useful in real settings.

mks0601 commented 2 years ago

Actually, the different color spaces are applicable to all motion capture datasets, such as Human3.6M. The motion capture datasets provide accurate GT 3D pose, but have limited or different image appearance. The in-the-wild datasets, such as MSCOCO, have GT 2D pose (no 3D pose), but have diverse and realistic image appearance. There are trade-offs between two types of dataset and you should jointly train a 3D hand pose estimator on both types of dataset to get benefit of two types of datsets.

mks0601 commented 2 years ago

Simply changing colors of hand skins cannot solve the image appearance domain gap between this dataset and real images because of backgrounds, motion blurs of real images, and so on.

mks0601 commented 2 years ago

I'm working on my new project for 3D interacting hand pose estimation in the wild. Please stay tuned.

rohitdavas commented 2 years ago

Thanks. would love to hear from your hand pose estimation in the wild project.

amundra15 commented 1 year ago

Are the color calibration parameters of the cameras available?

mks0601 commented 1 year ago

Sorry I do not have :(

amundra15 commented 1 year ago

I see. And the radial distortion parameters?

mks0601 commented 1 year ago

That neither :(