search

zc-alexfan / arctic

[CVPR 2023] Official repository for downloading, processing, visualizing, and training models on the ARCTIC dataset.
https://arctic.is.tue.mpg.de
Other
314 stars 20 forks source link

Hand-Object Tracking #33

Closed samueleruffino99 closed 11 months ago

samueleruffino99 commented 11 months ago

Are you planning to setup a pipeline for Object tracking task? Moreover, do you think ARCTIC would be a good benchmark for object tracking (using hand motion prior in some way)? If so, how could I get 2D objects' bounding boxes extracting from the poses? Thank you and great work!

zc-alexfan commented 11 months ago

Hi, we don't have plan to create an object tracking task from hand motion. But I had a few ideas on object pose estimation from hand pose. I think you can still learn a method to track object poses from hand poses for ARCTIC objects.

Could you clarify what you meant by "object 2D bounding box"?

samueleruffino99 commented 11 months ago

Maybe I will try to improve the object pose attending some information on hand poses (both left and right) and vice-versa. With "object 2D boxes" I am meaning the 2D projections of object onto the image plane.

zc-alexfan commented 11 months ago

I see. The preprocessing code already projected 3d vertices of the object to 2d before defining the 2d bounding boxes (see here.

So I suggest to check out our preprocessing script here. See here for information on how to run it.