How to implement "clean" and other primitive actions for behavior-100

[whcpumpkin]

Dear authors, When I was completing the behavior benchmark, I found a task that required cleaning some dust or stains. However, according to the iGibson 2.0 paper, this involves using tools like mops to interact with the stains. I am wondering if there is an primitive action available for this process (similar to grasp or navigate_to ). Alternatively, is it possible to directly modify the state of the stains within the simulator? Similarly, for actions like slicing or toggle_On, are there primitive actions available or can the state be directly modified?

[ChengshuLi]

Great question!

Currently, there isn't a primitive action available for wiping/toggling/slicing. However, you can implement them if you assume ground truth information from the simulator. For example, for toggling, you can access the toggle button position and command the robot's end-effector to reach that position. For slicing and wiping, the robot can first navigate to and grasp the first object (e.g. a knife or a towel), and then navigate to and approach the second object (e.g. a fruit to slice or a table surface to wipe). These primitives are a little bit trickier to implement than grasp or navigate to, but in theory they are possible.

Alternatively, it is possible to directly modify the object states. For example, you can do table.states[Stained].set_value(False), which will immediately remove all the stain particles on the table. So if you are only interested in high-level planning and don't care too much about low-level robot execution, you can call the setter directly. Same applies to all object states, including sliced, toggled on and so on.

[whcpumpkin]

Great question!

Currently, there isn't a primitive action available for wiping/toggling/slicing. However, you can implement them if you assume ground truth information from the simulator. For example, for toggling, you can access the toggle button position and command the robot's end-effector to reach that position. For slicing and wiping, the robot can first navigate to and grasp the first object (e.g. a knife or a towel), and then navigate to and approach the second object (e.g. a fruit to slice or a table surface to wipe). These primitives are a little bit trickier to implement than grasp or navigate to, but in theory they are possible.

Alternatively, it is possible to directly modify the object states. For example, you can do table.states[Stained].set_value(False), which will immediately remove all the stain particles on the table. So if you are only interested in high-level planning and don't care too much about low-level robot execution, you can call the setter directly. Same applies to all object states, including sliced, toggled on and so on.

Many thanks!

[whcpumpkin]

Hi! I have one more question: After modifying the object's state in the simulator, how can I make the modifications reflect in the rendering within the simulator as well? for example, if I want to open the sink to get an object soaked, what should I do? In my implementation, I first toggle_on a sink:sink.states[ToggleOn]._set_value(True) Then, manually update the sink state: sink.states[WaterSource]._update() Finally, set the object soaked: object.states[Soaked]._set_value(True)

However, in the saved image, I cannot see any water running in the sink. I'm wondering if there's an issue with my process or if there's a specific way to manually update the status in the simulator.

[whcpumpkin]

I also found that the success rate of the provided primitive action "grasp" is quite low, is there any solution such as "magic grasp" to improve the success rate of grasp?

magic grasp refers to only detecting the distance condition and then directly moving the object into the robot's hand. But in the iGibson code, the class igibson.object_states.robot_related_states.InHandOfRobot can not use_set_value() to grasp an object in the simulator.

Could the authors give some help about "grasp"? @ChengshuLi

[cgokmen]

You can teleport the object to the robot's hand position instead of physically executing a grasp, e.g. obj.set_position.orientation(*robot.eef_links["right_hand"].get_position_orientation()) . For this to work well the end effector should be in "sticky" grasping mode rather than assisted or physical, which you can set in the config file you are using.

Re: the soaking, you can toggle on the sink and put the towel underneath, then step the simulation for a second. Water will drop out and the towel will get wet. Generally, the internal functions starting with an underscore (especially the _update function) are meant to be called by the simulator or some other internal part and they will not work well if you call them manually.

sink.object_states[ToggledOn].set_value(True)
rag.set_position(sink.object_states[WaterSource].get_link_position() - np.array([0, 0, -0.2]))
for _ in range(100):
    simulator.step()

[whcpumpkin]

You can teleport the object to the robot's hand position instead of physically executing a grasp, e.g. obj.set_position.orientation(*robot.eef_links["right_hand"].get_position_orientation()) . For this to work well the end effector should be in "sticky" grasping mode rather than assisted or physical, which you can set in the config file you are using.

Re: the soaking, you can toggle on the sink and put the towel underneath, then step the simulation for a second. Water will drop out and the towel will get wet. Generally, the internal functions starting with an underscore (especially the _update function) are meant to be called by the simulator or some other internal part and they will not work well if you call them manually.

sink.object_states[ToggledOn].set_value(True)
rag.set_position(sink.object_states[WaterSource].get_link_position() - np.array([0, 0, -0.2]))
for _ in range(100):
    simulator.step()

Thank you so much for your help.