search

xukechun / Vision-Language-Grasping

[ICRA 2023] A Joint Modeling of Vision-Language-Action for Target-oriented Grasping in Clutter
73 stars 8 forks source link

Supervise the grasp pose of "tasks" #6

Closed lif314 closed 7 months ago

lif314 commented 7 months ago

How to supervise the grasping of "tasks"? For example, if I want "the cup", my task is to "pour" (task) water, so the part I want to grasp should be the cup handle, not other parts of the cup. I'm very confused about this point!

xukechun commented 7 months ago

Hi,

This problem is also interesting, namely task-oriented grasping (TOG), where the robot should reason the grasping part of the object for the following task. Our work lies in the concept of target-oriented grasping, which aims to successfully grasp the object specified by the language, and does not consider the following task. Now there are some datasets for task-oriented grasp, e.g. TaskGrasp, GraspGPT. Also, LLM is a good tool to generate the grasping part of the same object according to different tasks. Hope this helps you.

Best, Kechun Xu

lif314 commented 7 months ago

Hi,

This problem is also interesting, namely task-oriented grasping (TOG), where the robot should reason the grasping part of the object for the following task. Our work lies in the concept of target-oriented grasping, which aims to successfully grasp the object specified by the language, and does not consider the following task. Now there are some datasets for task-oriented grasp, e.g. TaskGrasp, GraspGPT. Also, LLM is a good tool to generate the grasping part of the same object according to different tasks. Hope this helps you.

Best, Kechun Xu

Thank you for your reply!