isaac-sim / IsaacLab

Unified framework for robot learning built on NVIDIA Isaac Sim
https://isaac-sim.github.io/IsaacLab
Other
2.22k stars 916 forks source link

[Question] Checking Collision between Object and Franka Robot #97

Closed LenaP7300 closed 1 month ago

LenaP7300 commented 1 year ago

Hello,

I am searching for a function to check for collision between the Franka Robot and a Rigid Object. More concretely, a function that tells the Environment when a collision has occurred. I could not find any such function under omni.isaac.robot or omni.isaac.objects.rigid. Am I looking in the wrong place or missing something, or does something like this not exist?

Best regards.

Mayankm96 commented 1 year ago

You can use the RigidContactView from Isaac Sim for this: https://docs.omniverse.nvidia.com/isaacsim/latest/ext_omni_isaac_core.html#create-rigidcontactview

LenaP7300 commented 1 year ago

Thank you very much for the tip! I tested a few things based on your example. The RigidObject has a RigidPrimView cylinder.objects. I called cylinder.objects.get_net_contact_forces() after setting track_contact_forces=True. (I had to adjust the implementation of cylinder.objects for this to work.) I now get a valid matrix, but whenever the robot hits the cylinder this error appears:

2023-07-20 19:47:29 [19,072ms] [Warning] [omni.hydra.scene_delegate.plugin] InstanceAdapter - cannot find cache item for proto /World/envs/env_0/Cylinder/collisions.proto_collisions_id0 2023-07-20 19:47:29 [19,072ms] [Warning] [omni.hydra.scene_delegate.plugin] InstanceAdapter - cannot find cache item for proto /World/envs/env_0/Cylinder/visuals.proto_visuals_id0

The Franka Robot then glitches into the table.

If I use more than one environment self.cylinder.initialize(self.env_ns + "/.*/Cylinder", track_contact_forces=True, prepare_contact_sensors=True) raises the exception "Failed to set rigid body transforms in backend"

Both faults only appear when track_contact_forces is true, while everything else remains unchanged. If you have any ideas on how I could solve this, I would appreciate them. Otherwise, thank you for your time!

Mayankm96 commented 1 year ago

Is it possible to get a repro for this? Maybe it is something to do with the tuning of collision offsets.

LenaP7300 commented 1 year ago

The errors I am getting are the same as in this post . You can find the code for the environment in which the errors appear here.

2361098148 commented 1 year ago

I am facing the same problem, Have you figure out how to solve the problem?

LenaP7300 commented 1 year ago

I figured it out. I applied the ContactReporter in the function initialize_views, but I think that it creates a prim or something. Orbit seems to have some trouble adding them anywhere other than design_scene. As soon as I added it in design_scene, it worked.

2361098148 commented 1 year ago

Could you share the gist, I would appreciate. Do you mean add self._initialize_views() here:

    def _design_scene(self) -> List[str]:
        # ground plane
        if self.cfg.terrain.use_default_ground_plane:
            # use the default ground plane
            kit_utils.create_ground_plane(
                "/World/defaultGroundPlane",
                static_friction=1.0,
                dynamic_friction=1.0,
                restitution=0.0,
                improve_patch_friction=True,
                combine_mode="max",
            )
        else:
            prim_utils.create_prim("/World/defaultGroundPlane", usd_path=self.cfg.terrain.usd_path)

        # robot
        self.robot.spawn(self.template_env_ns + "/Robot")

       # initial views
       self._initialize_views()

But the errors are still. Could you tell me how to add it into design_scene? I'm not quite sure yet. Whats more, I want to check the contact force between the legged robots' feet and the ground

2361098148 commented 1 year ago

RigidContactView Please, could you give me a hand. I have tried but failed, could you give me more details. I tried play_quadruped, And

feet_body = RigidPrimView(
prim_paths_expr=f"{prim_paths_expr}/{body_name}", name=foot_name, reset_xform_properties=False,
track_contact_forces=True, prepare_contact_sensors=True
)

But I only get zero tensor when i use the get_net_contact_forces()

LenaP7300 commented 1 year ago

In init()

subscribe to physics contact report event, this callback issued after each simulation step

    self._contact_report_sub = get_physx_simulation_interface().subscribe_contact_report_events(self._on_contact_report_event)

def _design_scene(self):

ground plane

    kit_utils.create_ground_plane("/World/defaultGroundPlane", z_position=-1.05)
    # table
    prim_utils.create_prim(self.template_env_ns + "/Table", usd_path=self.cfg.table.usd_path)
    # cube
    self.cube.spawn(self.template_env_ns + "/Cube")
    cubePrim = prim_utils.get_prim_at_path(self.template_env_ns + "/Cube")
    PhysxSchema.PhysxContactReportAPI.Apply(cubePrim)
    # robot
    self.robot.spawn(self.template_env_ns + "/Robot")
    # camera
    self.camera.spawn(self.template_env_ns + "/Camera", translation=[1.5, 1.5, 1.5], orientation=[0.0, 0.0, 0.0, 0.0])
    set_camera_view([3.0, 1.0, 1.3], [0.0, 0.0, 0.0], self.camera.prim_path)

    # setup debug visualization
    if self.cfg.viewer.debug_vis and self.enable_render:
        # create point instancer to visualize the goal points
        self._goal_markers = PointMarker("/Visuals/ee_goal", self.num_envs, radius=0.05)
        # create marker for viewing end-effector pose
        self._ee_markers = StaticMarker(
            "/Visuals/ee_current", self.num_envs, usd_path=self.cfg.marker.usd_path, scale=self.cfg.marker.scale
        )
        # create marker for viewing command (if task-space controller is used)
        if self.cfg.control.control_type == "inverse_kinematics":
            self._cmd_markers = StaticMarker(
                "/Visuals/ik_command", self.num_envs, usd_path=self.cfg.marker.usd_path, scale=self.cfg.marker.scale
            )
    # return list of global prims
    return ["/World/defaultGroundPlane"]

I did not end up using RigidPrimView, but the ContactReporter as in the example here. I hope this helps.

2361098148 commented 1 year ago

Thanks for your reply! My problem is solved by the method proposed in issue110, finally.

notFoundThisPerson commented 4 months ago

Hello, is there any API that reports which object is in collision with the robot? The RigidContactView seems only returns the contact force vector of the links without the object information, e.g. I want to check whether the specified cube is in collsion with the robot, rather than something else.