[Question] Gravity compensation

The documentation says the following about gravity compensation for velocity control:

While performing velocity control, in many cases, gravity compensation is required to ensure better tracking of the command. In this case, we suggest disabling gravity for the links in the articulation in simulation.

I am using a velocity-based IK controller. While disabling gravity works for just a robot arm in itself, issues arise when the robot picks up anything that has a mass. E.g. pick and place tasks are thus not really possible without the end-effector being constantly dragged down.

I am wondering how I could do gravity compensation including the payload. Since Isaac Sim already does physics simulations, I thought it maybe should be possible to retrieve what external forces are acting on the bodies of the robot, and apply a counter-acting force directly on these bodies. I did not manage to find this exposed anywhere in the Orbit API's (I could only find how to retrieve applied joint forces), but I did find the following method for setting external forces and torques: https://isaac-orbit.github.io/orbit/source/api/orbit/omni.isaac.orbit.assets.html#omni.isaac.orbit.assets.RigidObject.set_external_force_and_torque

Unfortunately though, this did not change the interaction when picking up an object. Maybe I am using it wrong?

self.robot.set_external_force_and_torque(forces=torch.zeros(0, 3, device=sim.device), torques=torch.zeros(0, 3, device=sim.device))
self.robot.write_data_to_sim()
self.sim.step()

I have tried to find examples where gravity is compensated, but the examples either turn off gravity and do not interact with objects, or they only seem to work for reinforcement learning (e.g. Isaac-Lift-Cube-Franka-v0) .

How could I correctly compensate for gravity not just for the robot but also for the payload? Would set_external_force_and_torque() be a good use for this? Or would there be any alternative approaches?

isaac-sim / IsaacLab

[Question] Gravity compensation #211