Handle moves within door plane

[alex-mitrevski]

Hi,

We are trying to use the environment for training a Toyota HSR to open doors, but we seem to have some problem with the collisions, which leads to the handle moving within the plane of the door. The problem is illustrated in the image below, where you can see the handle being pushed away towards the middle of the door:

In case it's relevant, we generated our robot model from the HSR's URDF (if it helps to debug the issue, we have the model in our fork: https://github.com/alex-mitrevski/DoorGym/blob/feature/hsr/world_generator/robot/hsr.xml). For the collision, we used contype="16" and conaffinity="7"as in the Blue floating hook model.

We trained the robot using PPO and the training seemed to be going well, but the learned policy obviously makes no sense due to the incorrect physical behaviour.

Do you by any chance know what might be going wrong here? Are we perhaps making a mistake in defining the collisions?

Thanks!

[yusukeurakami]

@alex-mitrevski Sorry for the late response.

Please change the 648th line of "world_generator.py" and try it again. damping=30000 -> damping=300000

    ######## Level3 body (Knob Wrapper) ########
    body3_joints = []

    axes = [[0,1,0],[0,0,1]]
    ranges = [[-0.2, 0.3], [-0.5,0.5]]
    for i in range(2):
        body3_joints.append(
        e.Joint(
            name='target{}'.format(i),
            type='slide',
            axis=axes[i],
            armature=0,
            stiffness=0,
            damping=300000,
            frictionloss=0,
            limited="true",
            range=ranges[i]))

The problem happens because the doorknob is located on a door by slide joints on the door and not completely fixed. We intentionally did that in order to randomzie the position of the doorknob at every iteration (It randomized the y,z location on a door plane.) We've set the the sliding joints' ("target0", "target1") damping value pretty high (30000) to prevent the doorknob to be moved, but it seems like it wasn't enough in your case. Therefore, try again with higher damping value (300000) for a doorknob.

Tell me what happened after doing this.

[alex-mitrevski]

@yusukeurakami Sorry for only replying now.

Increasing the damping did indeed fix the issue with the moving doorknob; thanks for that! I additionally had to change the allowed angles for the handle to [-knob_rot_range, 0] though; otherwise, the robot was learning to push the handle up rather than press on it. Eventually, our robot managed to learn that it needs to push down on the handle, but unfortunately didn't learn that it also needs to pull so that the door opens.

I also noticed that the springiness of the handle is not ideal, i.e. if the robot pushes down and then retracts, the handle doesn't seem to go back up. I did play around with the parameters a little bit, but that's still a remaining problem.

[yusukeurakami]

@alex-mitrevski Thank you for your reply. Glad to hear that damping fixed the problem. Whether the robot learns to pull the door or not was one of the problems we got as well. We needed some reward engineering (e.g. give more reward for opening a door instead of turning a knob.), and also a tuning of the right randomness of the environments.