search

facebookresearch / InterHand2.6M

Official PyTorch implementation of "InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image", ECCV 2020
Other
676 stars 92 forks source link

Question about using my own image to test #19

Open kingsman0000 opened 3 years ago

kingsman0000 commented 3 years ago

1.Hello,first I have question about the result of using my own image to test,the result seems not really good.Especially the Little finger part,it didn't catch well as the image I show.However,while I'm using your InterHand2.6M datasets,the result seems really good,so I'm confuse about is the model's problem or not. mine mine2 mine3 2.Second,I want to ask how can I use particular folder include my own image to test.Because now I need to follow the sequence of annotation json file.Thanks!

mks0601 commented 3 years ago

InterHand2.6M dataset is the first dataset that contains 3D interacting hand poses, however it has several limitations including the image appearance. As you can see, all the images are captured in the multi-view studio with black backgrounds. Thus, a model trained solely on our dataset may be generalize well to some in-the-wild images. I recommend to mix some 2D in-the-wild datasets such as MSCOCO (https://github.com/jin-s13/COCO-WholeBody).

Regarding the second question, I cannot understand your question :(

kingsman0000 commented 3 years ago

InterHand2.6M dataset is the first dataset that contains 3D interacting hand poses, however it has several limitations including the image appearance. As you can see, all the images are captured in the multi-view studio with black backgrounds. Thus, a model trained solely on our dataset may be generalize well to some in-the-wild images. I recommend to mix some 2D in-the-wild datasets such as MSCOCO (https://github.com/jin-s13/COCO-WholeBody).

Regarding the second question, I cannot understand your question :(

Thanks for you reply!What I mean about second question is I don't want to use InterHand2.6M dataset to predict,however,I want to use my own image to test.But now I can only remove original data (InterHand2.6M) according to annotations file's sequence.like:cam400262,cam400269,cam400271.....and so on. For example:I only have one image to test but I need to predict all val or test data repeat and it's really waste time.So I confuse about if I can only predict for my particular image(I can put my image in some separate folder).Thanks!

mks0601 commented 3 years ago

You can just create a new dataloader. As you can see, separated dataloaders are defined for each dataset in data/$DB_NAMNE/$DB_NAME.py. You can make a new dataloader of yours.

saqib22 commented 3 years ago

@kingsman0000 Did you use inv_trans in your testing ? Thank You !

kingsman0000 commented 3 years ago

@kingsman0000 Did you use inv_trans in your testing ? Thank You !

I only remove original dataset(InterHand2.6M) and use my own image to predict.I'm follow the original code and it include inv_trans.

mks0601 commented 3 years ago

Hi guys. I added a demo code for a random image.

kingsman0000 commented 3 years ago

Hi guys. I added a demo code for a random image.

Thank you very much!However,the result seems not very good for your interhand2.6M dataset or my own image.Is there any wrong for the model? result_2d result_2d_1 result_2d_2 result_2d_3 result_2d_4

mks0601 commented 3 years ago

I think you didn't set proper bbox in here? Let me add that in the instruction.

kingsman0000 commented 3 years ago

I think you didn't set proper bbox in here? Let me add that in the instruction.

Thanks I didn't notice that part.Then how can I get testing image's proper bbox(xmin, ymin)because I only Know about width and height about the image.

mks0601 commented 3 years ago

width and height means width and height of bbox. Maybe you can train a hand detector using detectron2.

kingsman0000 commented 3 years ago

So what you mean is that I need to train a model to get my own hand's bbox by using detectron2 just like you suggest？ And how about your pre-train model (snapshot_20.pth.tar)for use?Is that model only for interhand2.6M dataset or do I still need to change suitable bbox about each image(interhand2.6M dataset)?Thanks.

mks0601 commented 3 years ago

The pretraiend model takes a cropped and resized hand image. So you should detect the hand using off-the-shelf hand detector first, and then apply the pretrained model.