search

simpler-env / SimplerEnv

Evaluating and reproducing real-world robot manipulation policies (e.g., RT-1, RT-1-X, Octo) in simulation under common setups (e.g., Google Robot, WidowX+Bridge) (CoRL 2024)
https://simpler-env.github.io/
MIT License
326 stars 43 forks source link

how to make camera rotate around some objects in scene #51

Open Lucky-Light-Sun opened 6 days ago

Lucky-Light-Sun commented 6 days ago

Hello, I'm new to the robotics domain and have just started exploring SimplerEnv. I find it is an really impressive work! Currently, I'm doing some experiments using SimplerEnv.

To be specific, I wanna to rotate the camera around some points in SimplerEnv. These points can be some objects in scene or some meaningless points. So I can test Octo/OpenVLA models' generalizability to view point. For example, I run scripts/octo_pick_coke_can_variant_agg.sh code, and get the scene shown below:

image

In this scene, Q1.how can I get the coke can coordinate relative to the world/robot coordinate system? I may use it as rotation center. Q2.how can I get the camera coordination frame(position, rotation) relative to the world/robot coordinate system.

So with these coordinations information, I can make camera rotate around some points.

What'more, in Q2, should I use forward kinematics to calculate the relative coordinate? But I have no idea how to get the relative frame rotation for each robot joints.

Could you please help me to solve these problems? Thank you again for your contributions to the community!

Looking forward to your reply!!

xuanlinli17 commented 6 days ago

Q1: You can checkout the environment source code at e.g., https://github.com/simpler-env/ManiSkill2_real2sim/blob/main/mani_skill2_real2sim/envs/custom_scenes/grasp_single_in_scene.py and you can get the object pose wrt world by env.obj.pose Q2: The camera pose wrt world frame can be directly obtained through e.g., https://github.com/simpler-env/ManiSkill2_real2sim/blob/main/mani_skill2_real2sim/examples/demo_manual_control_custom_envs.py#L253C13-L253C66 . You don't need to perform forward kinematics.

Lucky-Light-Sun commented 6 days ago

Q1: You can checkout the environment source code at e.g., https://github.com/simpler-env/ManiSkill2_real2sim/blob/main/mani_skill2_real2sim/envs/custom_scenes/grasp_single_in_scene.py and you can get the object pose wrt world by env.obj.pose Q2: The camera pose wrt world frame can be directly obtained through e.g., https://github.com/simpler-env/ManiSkill2_real2sim/blob/main/mani_skill2_real2sim/examples/demo_manual_control_custom_envs.py#L253C13-L253C66 . You don't need to perform forward kinematics.

Thanks for your timely reply!!! I'll try it.