[Bug Report] DifferentialIKController not working correctly when applying rotation to Articulation

samueledelloca commented 5 months ago

Describe the bug

I'm developing a custom environment which makes (also) use of the DifferentialIKController for reaching a specific pose, defined through a command. Everything works fine until I apply a rotation into the Articulation's init_state. It took me a while to understand the cause, because the Articulation's behavior is pretty strange. See the attached example videos for both my custom environment and Isaac-Lift-Cube-Franka-IK-Abs-v0 task from orbit_tasks/manipulation/lift. I tried with different articulations and configurations but the error persists, even when using only RL and DifferentialInverseKinematicsAction without state machine. Other actions are working fine when rotating the Articulation. So I suppose that the initial rotation is not taken into account when computing the change in joint positions through the IK controller, but I'm not sure how to fix it.

Steps to reproduce

Simply add the following lines into FRANKA_PANDA_HIGH_PD_CFG and execute the Isaac-Lift-Cube-Franka-IK-Abs-v0 task with ./orbit.sh -p source/standalone/environments/state_machine/lift_cube_sm.py --num_envs 8. Note that this task aims first at reaching the cube object and then lifting it to a specific pose. I'm interested only in the second part, so I didn't fixed the code for reaching the cube after applying changes to initial position and rotation.

Additional context

You will find 2 attached videos: custom_environment (I apologize for the quality but it's the best I can upload) which shows the bug in my custom environment and franka_lift which is related to Isaac-Lift-Cube-Franka-IK-Abs-v0 and can be reproduced following the instructions in the previous section. The latter video is divided into 3 parts: 1- correct behavior without applying rotation 2- wrong behavior by applying rotation and translation 3- correct behavior by applying only translation

While for the custom environment video you have: 1- correct behavior without applying rotation 2- wrong behavior by applying rotation and translation

Checklist

[X] I have checked that there is no similar issue in the repo
[X] I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Acceptance Criteria

[ ] Articulation behaving correctly (i.e. by reaching the defined pose) using DifferentialIKController when applying an initial rotation

Mayankm96 commented 5 months ago

Are the targets given in the same frame as in which the controller is working?

samueledelloca commented 5 months ago

Are the targets given in the same frame as in which the controller is working?

Yes, I'm pretty sure they are, given that the target frames are visualized correctly as you can see in the videos. They are even reached correctly if I do not apply any initial rotation to the Articulation.

isaac-sim / IsaacLab