xuanlinli17
commented
2 weeks ago Hi,
The "odd interactions" are due to the robot arm bumping into the robot camera link and causes the camera link's pose to change. Since we overlay a fixed real-world background image onto the simulation observation, i.e., only the foreground robot and objects are rendered, and the background image will remain the same no matter what the robot's camera pose is, this causes "weird observations" in the video.
For the environment speed, we use 500hz simulation frequency (i.e., # simulation steps / second) and 3hz robot control (action) frequency for the google robot tasks. If I understand correctly, the 12fps you refer to is the number of robot actions / second. Note that the 3hz control frequency is a lot lower than the commonly-used ones e.g., 20hz when benchmarking simulation speed. Also, the rendered image's resolution is much higher (640x512 for google robot) compared to simulation speed tests that render 128x128 image. Thus simulation "slow down" is normal.
Though, once the ManiSkill3 environment is ready, the simulation speed will be much faster and also with GPU parallelizations.
Hello, thank you for providing this project!
I have been using this environment to train SAC and PPO agents and have noticed some odd interactions with the environment like the one attached below.
I have also noticed that in my case, the environment runs at about 12 FPS whereas maniskill can achieve roughly 350 FPS. Is this normal? Looking forward to your reply!