[Question] Pre-assembled Franka Panda + Robotiq gripper provided ? Issues with self assembled Franka panda + Robotiq gripper.

[johnny-wang16]

Question

I'm new to Isaaclab and was wondering if there are any pre-assembled files provided for Franka Panda + Robotiq 2f 85 gripper? I’ve seen the tutorials on rigging robots with robotiq gripper but it seemed out of date and unclear. I’ve also found broken robotiq 2f85 on omniverse server which is not that useful. I just need this robot configuration for reinforcement learning training. Please advise. Thanks!

To fix the gripper, I follow this instruction and this youtube video. The resulting gripper I got is here and it seemed functional when applying force on to it. The franka file I used it the "franka_instanceable.usd" in here.

Now that the gripper seemed fixed, I tried to connect them together with the robot assembler tool provided in Isaac Utils but it does not work. The robot just flies everywhere for some reason.

Finally, I followed this tutorial on assembling gripper on arm. Some details are not clear so I put my steps here: 1) "Open" the franka_instanceable.usd so that I'll be able to delete the fingers and hand (instead of deactivate).

2) Drag and drop 2f85_instanceable.usd in this folder to "Layer". So layer structure looks like the image below. I think this allows me to delete the root_joint later on and change the transform of the gripper but im not sure if this is correct. image

steps 3) to 5) Follow the 3 steps in this tutorial.

6) I set damping and stiffness of the joints on robotiq gripper to 10 just for testing. 7) Drag robotiq gripper to after panda_link7 (not under). 8) Set franka_instanceable as default prim. 9) delete the articulation root under robotiq_arg2f_85_model.

After all these steps, I tried running it but still faced several problems: 1) The gripper acts weird when simulation starts. It first tries to close and eventually opens up. This is not desirable. This behavior can be seen from the video in 1). I suspect it might be due to the damping and stiffness parameter but not sure. Or maybe it's some pre-programmed trajectory that I'm unaware of. Video here.

2) The actual actuated joints in the gripper are right/left "outer_knuckle_joint" but when I print the joint_names of the Articulation object, it gives me other joints that does not even have a drive component in them like: 'right_inner_finger_joint', 'left_inner_finger_joint', 'RevoluteJoint', 'RevoluteJoint' . This is annoying because now my action space is unnecessarily larger. Is there a way to fix this such that, joint_names only return actuated joints with drive? why RevoluteJoints that aren't actuated is included as joints in robot? Testing code for this part can be found here.

Here's my assembled USD file in Collected_franka_robotiqnamed franka_robotiq.usd. Any advice would be greatly appreciated. Thank you very much for reading my problem.

Update on 1): I think it has to do with the stiffness and damping parameters in the gripper joints. Somehow when i made a new one, the weird behavior goes away. I still don't know why the robot follows that particular motion when playing tho.

[jsmith-bdai]

First of all wanted to applaud your resourcefulness - glad you found a way to build up the Franka + 2F85 gripper!

Seems like you have resolved 1., so I can comment on 2.

Your action space shouldn't be directly determined from the Articulation definition - this is specified in your ActionCfg. For the Cartpole example (https://isaac-sim.github.io/IsaacLab/main/source/tutorials/03_envs/create_manager_base_env.html#defining-actions) you'll see that while the cartpole has 2 joints, only the slider_to_cart joint is controlled. You can similarly configure your ActionCfg to only control those joints that you wish.

Can you share your EnvCfg so I can take a look and make suggestions? I have a feeling you might be using * wildcard to select all joints for you action space.

[johnny-wang16]

Thanks for your response! My custom config is a DirectRL environment though. This is my config:

@configclass
class FrankaCustomEnvCfg(DirectRLEnvCfg):
    # env
    episode_length_s = 8.3333  # 500 timesteps
    decimation = 2
    num_actions = 13
    num_observations = 23
    num_states = 0

    # simulation
    sim: SimulationCfg = SimulationCfg(
        dt=1 / 120,
        render_interval=decimation,
        disable_contact_processing=True,
        physics_material=sim_utils.RigidBodyMaterialCfg(
            friction_combine_mode="multiply",
            restitution_combine_mode="multiply",
            static_friction=1.0,
            dynamic_friction=1.0,
            restitution=0.0,
        ),
    )

    # scene
    scene: InteractiveSceneCfg = InteractiveSceneCfg(num_envs=4096, env_spacing=3.0, replicate_physics=True)

    # robot
    robot = ArticulationCfg(
        prim_path="/World/envs/env_.*/Robot",
        spawn=sim_utils.UsdFileCfg(
            usd_path=f"./Collected_franka_robotiq/franka_robotiq.usd",
            activate_contact_sensors=False,
            rigid_props=sim_utils.RigidBodyPropertiesCfg(
                disable_gravity=False,
                max_depenetration_velocity=5.0,
            ),
            articulation_props=sim_utils.ArticulationRootPropertiesCfg(
                enabled_self_collisions=False, solver_position_iteration_count=12, solver_velocity_iteration_count=1
            ),
        ),
        init_state=ArticulationCfg.InitialStateCfg(
            joint_pos={
                "panda_joint1": 1.157,
                "panda_joint2": -1.066,
                "panda_joint3": -0.155,
                "panda_joint4": -2.239,
                "panda_joint5": -1.841,
                "panda_joint6": 1.003,
                "panda_joint7": 0.469,
                # "panda_finger_joint.*": 0.035,
                "left_outer_knuckle_joint": 0.035,
                "right_outer_knuckle_joint": 0.035,
            },
            pos=(1.0, 0.0, 0.0),
            rot=(0.0, 0.0, 0.0, 1.0),
        ),
        actuators={
            "panda_shoulder": ImplicitActuatorCfg(
                joint_names_expr=["panda_joint[1-4]"],
                effort_limit=87.0,
                velocity_limit=2.175,
                stiffness=200.0,
                damping=4.0,
            ),
            "panda_forearm": ImplicitActuatorCfg(
                joint_names_expr=["panda_joint[5-7]"],
                effort_limit=12.0,
                velocity_limit=2.61,
                stiffness=200.0,
                damping=4.0,
            ),
            "robotiq_hand": ImplicitActuatorCfg(
                joint_names_expr=["left_outer_knuckle_joint", "right_outer_knuckle_joint"],
                effort_limit=200.0,
                velocity_limit=0.2,
                stiffness=2e3,
                damping=1e2,
            ),
        },
    )

[jsmith-bdai]

Gotcha - didn't realize you were in Direct workflow, my bad. I re-read your question and realize you just want to be able to acquire non-articulated joints from joint_names?

The way we've defined joint names is to represent all of the joints (regardless of whether they are actuated). I think a workaround for you would be to hardcode the actuated joints e.g. ["left_outer_knuckle_joint", "right_outer_knuckle_joint"] I can also check with the reset of the dev team to see if it makes sense to implement an actuated_joint_names property.

[johnny-wang16]

Yes, i just want to be able to acquire non-articulated joints from joint_names. It's honestly not a big problem but i personally feel there should be a way to directly access the actuated ones. Otherwise, to set the joint positions using the set_joint_position_target method, I'll have to pass in some dummy values for the unactuated joints which is non ideal.