[Question]How can I get real-time bbox about object when its position changes in motionplanning?

[DevinQiao]

Hello, I'm having some problems.

1. Can I get real-time bbox for the cube? I lift the cube in the mani_skill/envs/tasks/tabletop/pick_cube.py file as follows:

   def _initialize_episode(self, env_idx: torch.Tensor, options: dict):
       ...
       xyz[:, 2] = self.cube_half_size + 0.5
       ...

   I run the mani_skill/examples/motionplanning/panda/run.py file but the robot cannot complete the task. It seems that the robotic arm is still trying to grasp the cube from its original position. The video below records the trajectory.

https://github.com/user-attachments/assets/01f14f11-a6b7-47c2-8b51-13516e34bb0f

2. Can I hold the robotic arm in one pose? I tried to add the following code in mani_skill/examples/motionplanning/panda/solutions/pick_cube.py to let the env reach a stable state. Then let the robot grasp the cube. But it's stuck in a loop.

   def solve(env: PickCubeEnv, seed=None, debug=False, vis=False):
   obs, _ = env.reset(seed=seed)
   while True:
       # action = env.action_space.sample()
       action = torch.zeros(1, 8)
       new_obs, _, _, _, _ = env.step(action)
       if torch.equal(obs, new_obs):
           break
       else:
           obs = new_obs
   planner = PandaArmMotionPlanningSolver()

[StoneT2000]

torch.equal probably is not a good idea, you should loosen the equality constraint more or else it won't break out of the loop.

Moreover spawning the cube as you did that high, might cause it to bounce off the robot and be too far away