search

facebookresearch / InterHand2.6M

Official PyTorch implementation of "InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image", ECCV 2020
Other
676 stars 92 forks source link

How can I visualize just like the demo video from my own video or in real-time? #7

Open kingsman0000 opened 3 years ago

kingsman0000 commented 3 years ago

or when will the code release thanks!

mks0601 commented 3 years ago

sorry visualizing on other videos will be supported later.

lisa676 commented 3 years ago

@mks0601 Is there any update about this issue (real-time testing)?

mks0601 commented 3 years ago

Sorry I'll be very busy until CVPR deadline (mid of Nov.) :(

mks0601 commented 3 years ago

Hi guys. I added a demo code for a random image.

shawnricecake commented 3 years ago

Hi, can you share which model you used in that demo for the given picture? the InterHand H or M or H+M?

mks0601 commented 3 years ago

The link to the pre-trained model in the demo section is trained on H+M. Basically, H+M is always better than H and M.

shawnricecake commented 3 years ago

The link to the pre-trained model in the demo section is trained on H+M. Basically, H+M is always better than H and M.

Thanks a lot for your response!

shawnricecake commented 3 years ago

The link to the pre-trained model in the demo section is trained on H+M. Basically, H+M is always better than H and M.

Hi, I have used the model based on interhand dataset with H+M on google drive, but I did not get the same result with you, I mean the picture you gave in demo file named 'result_2d.jpg'.

Is there anything wrong with the parameter of bbox? I just used the bbox = [69, 137, 165, 153]

By the way, what I got is below: git_result

mks0601 commented 3 years ago

That's weird... I got the same result like below. result_2d

Could you add bbox = np.array(bbox, dtype=np.float32) right after bbox = [69, 137, 165, 153]?

shawnricecake commented 3 years ago

That's weird... I got the same result like below. result_2d

Could you add bbox = np.array(bbox, dtype=np.float32) right after bbox = [69, 137, 165, 153]?

Hi, the result is same as I showed above.

Anyway, it is not important, it might be caused by some attribution of the picture.

By the way, I have converted your picture demo code into a video (camera) demo, and I have realized it in real-time. What I found is that, the model is good for the right hand, but for left hand, it is bad. Pictures below are some photos of my experiments.

image image image image image

If you have any recommendations, please let me know.

thanks for your response again!

mks0601 commented 3 years ago

This is weird. The accuracy of the right and left hand are almost the same. Could you tell me how did you change the code and could you provide your demo image?

shawnricecake commented 3 years ago

This is weird. The accuracy of the right and left hand are almost the same. Could you tell me how did you change the code and could you provide your demo image?

my camera demo code is not really different from your picture demo code, I just change the bbox into the whole pic, I mean:

original_img_height, original_img_width = original_img.shape[:2] bbox = [0, 0, original_img_height, original_img_width]

The reason is that I found it can bring better results when I put my hand in front of the camera and when the hand take the most area of the video (I mean when the background is white and the whole camera is filled with hand) it can get good results.

By the way, I just inference the picture and draw the picture with every frame of the camera, and the only parameter I changed was the bbox

mks0601 commented 3 years ago

I guess the reason is bbox. You should crop the hand area or if you want to set bbox as the whole image area, your hand must be close to the camera.

shawnricecake commented 3 years ago

I guess the reason is bbox. You should crop the hand area or if you want to set bbox as the whole image area, your hand must be close to the camera.

Yeah, I know that, but I am considering the reason that why it can get good result only with right hand....

shawnricecake commented 3 years ago

I guess the reason is bbox. You should crop the hand area or if you want to set bbox as the whole image area, your hand must be close to the camera.

Hi, I found where the problem is, I just forget that I have changed some parts of the net, but in the mean time, I still use your model ( ><, the code even not got an error when loading the weights....).

Now I have got the same demo result with you.

Thanks for your patience! Hope you safe and healthy!

mks0601 commented 3 years ago

Yes I disabled 'strict' option when loading the pre-trained weights (see here). I think I did this for the debugging purpose. Let me enable this option. Thanks!

Algabeno commented 3 years ago

@shen494157765 Hi,Can you share with me the camera demo code

vodanhbk95 commented 3 years ago

I guess the reason is bbox. You should crop the hand area or if you want to set bbox as the whole image area, your hand must be close to the camera.

Hi, I found where the problem is, I just forget that I have changed some parts of the net, but in the mean time, I still use your model ( ><, the code even not got an error when loading the weights....).

Now I have got the same demo result with you.

Thanks for your patience! Hope you safe and healthy!

Hi @shen494157765 , Can you tell me more about where the problem is and what you have fixed it. I am do the same thing like you: original_img_height, original_img_width = original_img.shape[:2] bbox = [0, 0, original_img_height, original_img_width] It got bad result. So please share me more.

shawnricecake commented 3 years ago

I guess the reason is bbox. You should crop the hand area or if you want to set bbox as the whole image area, your hand must be close to the camera.

Hi, I found where the problem is, I just forget that I have changed some parts of the net, but in the mean time, I still use your model ( ><, the code even not got an error when loading the weights....). Now I have got the same demo result with you. Thanks for your patience! Hope you safe and healthy!

Hi @shen494157765 , Can you tell me more about where the problem is and what you have fixed it. I am do the same thing like you: original_img_height, original_img_width = original_img.shape[:2] bbox = [0, 0, original_img_height, original_img_width] It got bad result. So please share me more.

Hi, the problem I had was not caused by the bbox, the problem was caused by the different model. As I said, I revised the original model, which means my model was different with the author's, but the old code still allows the revised model load the weights from original model.

shawnricecake commented 3 years ago

@shen494157765 Hi,Can you share with me the camera demo code

It is easy to write, what you need to do is just show every frame which is the demo picture result

ZhengdiYu commented 2 years ago

@shen494157765 Hi,Can you share with me the camera demo code

It is easy to write, what you need to do is just show every frame which is the demo picture result

Interesting. Did you split the network into two identical branches? I was wondering why would this happen. Because normally if you disable some part of the network, both two hands should be degenerated at the same time.