search

ZhengdiYu / Arbitrary-Hands-3D-Reconstruction

🔥(CVPR 2023) ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction
https://semanticdh.github.io/ACR/
MIT License
174 stars 12 forks source link

Some questions... #7

Closed haolyuan closed 1 year ago

haolyuan commented 1 year ago

Wonderful work! I want to know whether ACR could handle some occasions such as only one hand in image well, such as only one hand in image. Because I notice that the magic.jpg in demo folder is one hand. But I get some confusing results using video input. I wonder if there are some other settings such as img resizing or cropping. Testing resolution is 1280×720.

ZhengdiYu commented 1 year ago

Wonderful work! I want to know whether ACR could handle some occasions such as only one hand in image well, such as only one hand in image. Because I notice that the magic.jpg in demo folder is one hand. But I get some confusing results using video input. I wonder if there are some other settings such as img resizing or cropping. Testing resolution is 1280×720.

Thanks for your interest!

1) Yes, ACR could work perfectly under the one-hand, hand-object interaction scenario as shown in the teaser gif in README.md and videos. 2) Yes, all the input images are resized to 512x512 for batch processing and they maintain the same aspect ratio. Could you send or describe your 'confusing results' a bit more?

haolyuan commented 1 year ago

Like this image image It seems that detection meets something wrong

ZhengdiYu commented 1 year ago

Like this image image It seems that detection meets something wrong

This could happen when the --centermap_conf_thresh is a bit low. Please try to increase the threshold for detection. And please ensure to use the up-to-date pre-trained weights.

In the mean time, this looks unusual for a simple scene like this, could you maybe also send me the raw image or video? I could also try to re-produce your problem here.

haolyuan commented 1 year ago

Thanks! I will change the threshold and try again.

haolyuan commented 1 year ago

It seem increasing the threshold really work! But not for all images. Thanks a lot! image image Are the two hand detections seperated? It seems that left hand mesh changes little between two images under different thresholds.

ZhengdiYu commented 1 year ago

It seem increasing the threshold really work! But not for all images. Thanks a lot! image image Are the two hand detections seperated? It seems that left hand mesh changes little between two images under different thresholds.

I am glad to see that works. The reason for the changes should be that the cross-hand prior will be activated when there are two hands detected. In this case, when the right hand is actually missing but still being detected due to wrong detection & low threshold, the accidently predicted right hand will somehow disturb the left hand's performance.

haolyuan commented 1 year ago

I understand. Thanks!