isaac-sim / IsaacLab

Unified framework for robot learning built on NVIDIA Isaac Sim
https://isaac-sim.github.io/IsaacLab
Other
2.35k stars 980 forks source link

[Proposal] Get analytical Jacobian without simulation stepping #301

Closed catachiii closed 7 months ago

catachiii commented 8 months ago

Proposal

Calculate the analytical Jacobian (body pos/vel w.r.t generalized coordinates) without calling the simulation step function (with the imported URDF/USD).

Motivation

Analytical Jacobian is important for a wider range of robotic controls since they are fast and accurate. The method mentioned in Joonho's Science Robotics paper requires IK to work. The current estimation of the Jacobian is not accurate, and it is difficult to have exactly the same implementation on the C++ side for sim2real transfer.

Alternatives

There are also other libraries like Pinocchio trying to calculate the analytical Jacobian, but they need to be parallelized and could not be easily tested with Isaac Sim.

Additional context

Checklist

Acceptance Criteria

Add the criteria for which this task is considered done. If not known at issue creation time, you can add this once the issue is assigned.

Mayankm96 commented 8 months ago

We don't want to expose the Jacobians through the data field as this is an expensive quantity to fetch due to its size. You can use the root_physx_view to query from the Jacobians.

However, I think your question is more about "updating" these kinematic quantities without doing PhysX step. Unfortunately, this isn't possible. You can use Warp to make a sim model to query Jacobians. Please check their documentation on how to do so.

catachiii commented 8 months ago

Thanks @Mayankm96, I'll take a look!