search

med-air / DEX

[ICRA'23] Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robot
MIT License
32 stars 4 forks source link

Transform Needle Pose to World Frame #9

Open Homunculus314 opened 3 months ago

Homunculus314 commented 3 months ago

Hello,

I managed to export the trajectory and transform it from the PyBullet world frame to the RCM frame. I also managed to set the needle poses, but I have one final question.

When setting the needle position, it seems like I need to do this in the PyBullet frame. However, since I want to use the trajectory in a different simulation environment, I have to transform the desired pose of the needle to the PyBullet world frame first. I'm unsure how to do this transformation. I know how to transform my desired needle pose to the RCM frame though, but I don't know how to get to the PyBullet frame from there. The world2rcm function appears to work only for the robot arm. I would appreciate any suggestions.

TaoHuang13 commented 3 months ago

Hi,

This is a good question. The key point here is that we include some poses w.r.t. PyBullet frame in the RL observation. Currently, I do not have an idea how to include such poses in different simulation platforms or the real world.

I would suggest you exclude any variables wrt PyBullet frame in RL observation. Just keep the variable wrt RCM frame (or you may need to do additional transformation to get that). To do so, considerable modification should be performed. You may take a look at this function which defines the observation space of RL.

If the observation only depends on RCM frames, I think the transfer between different simulations will be smooth.