search

maxspahn / gym_envs_urdf

URDF environments for gym
https://maxspahn.github.io/gym_envs_urdf/
GNU General Public License v3.0
43 stars 14 forks source link

🔧 Fixed the Torque Input Issue for Torque Mode #231

Open behradkhadem opened 11 months ago

behradkhadem commented 11 months ago

I surfed the web and turns out the link you sent in https://github.com/maxspahn/gym_envs_urdf/issues/230#issuecomment-1708340979 had the correct solution. Now robots (I tested point and panda) react to torque input, but it seems that (at least in the case of panda arm) robot falls. By falling I mean that robot can't preserve the initial posture and the links kinda fall to the ground. It reacts to the input torque but with a disfigured position. Is it natural? We don't have this behavior in velocity mode.

I'd be happy if you test it.

maxspahn commented 11 months ago

hi @behradkhadem ,

great that you investigated that. If the robot is falling, that is because there is no gravity compensation. That makes totally sense. We could try to turn off gravity, but then obstacles also don't fall down.

The other thing, we can move the line you added to a more central place in the reset, so that it is not called in every time step.

maxspahn commented 11 months ago

By the way, there is already a method for disable the velocity control, like you did it. It is called disable_velocity_control.

maxspahn commented 11 months ago

Maybe we can solve #15 in this PR as well.

behradkhadem commented 11 months ago

We could try to turn off gravity

I think this is a bad idea. In that pick and place becomes impossible.

By the way, there is already a method for disable the velocity control, like you did it. It is called disable_velocity_control.

My bad! I didn't see it, sorry.

The other thing, we can move the line you added to a more central place in the reset, so that it is not called in every time step.

Would you elaborate on this matter please?

behradkhadem commented 11 months ago

Maybe we can solve #15 in this PR as well.

You propose that we have a singular friction for all joints (like the one you've implemented self._friction = 0.1) or we let user input an array of frictions for all joints (like this: [0.1, 0, 0, 0, 0.2, 0.05])?

maxspahn commented 11 months ago

Would you elaborate on this matter please?

Just use the function disable_velocity_control, so that we don't have to loop through the joints again.

You propose that we have a singular friction for all joints (like the one you've implemented self._friction = 0.1) or we let user input an array of frictions for all joints (like this: [0.1, 0, 0, 0, 0.2, 0.05])?

Could you use a default float value 0.1, and pass an optional argument somewhere, probably through a dedicated function set_friction where the friction can be set by a single float value or an array of values.

I think this is a bad idea. In that pick and place becomes impossible.

What is the alternative though?

behradkhadem commented 10 months ago

Could you use a default float value 0.1, and pass an optional argument somewhere, probably through a dedicated function set_friction where the friction can be set by a single float value or an array of values.

I'll work on it.

What is the alternative though?

I'm not in the position to suggest anything. Gravity compensation is a special and complicated matter for itself and there are papers that argue its importance in RL (like this for example). So, I don't know what to do. What is your opinion?

PS: Sorry for the delay in answering. Over the past few weeks, I've been dealing with serious health issues and wasn't able to focus on the project. I hope that I be able to cooperate on the project and make it better from now on.

Thanks.

maxspahn commented 10 months ago

I'm not in the position to suggest anything. Gravity compensation is a special and complicated matter for itself and there are papers that argue its importance in RL (like this for example). So, I don't know what to do. What is your opinion?

According to the paper, it is important that the RL agent should not learn the gravity compensation part. Could you try to use the inverse dynamics only for the gravity part and then add the action-torque to it?

behradkhadem commented 10 months ago

According to the paper, it is important that the RL agent should not learn the gravity compensation part. Could you try to use the inverse dynamics only for the gravity part and then add the action-torque to it?

I asked a few of my professors about this in past week and they either said they didn't know, or they said that in real life robots, also, we should deal with gravity compensation. So, the agent itself should learn to compensate the gravity. Although we can implement the compensator separately for simple robots, in complicated ones we will face challenges. So, to my limited understanding so far, it's better if the agent learns the gravity compensation part alongside with pick-and-place stuff. Apparently learning the compensation part can be facilitated with minimizing the energy used by each motor in a robot or penalizing the robot for using excessive torque. In which I'm not sure whether is it possible with data we have in our observation dict.

But the robot now responds to understandable amount of torque, correctly and we're good in that regard.

behradkhadem commented 10 months ago

I also added the friction torque input to the GenericRobot constructor (addressing this: https://github.com/maxspahn/gym_envs_urdf/issues/15). I tried adding it to the UrdfEnv but since we have a list of GenericRobot in each UrdfEnv, I thought it would be better if we add friction data of each robot inside the robot constructor itself. But adding the torque input to UrdfEnv is totally doable, we'll just need the robot id for each input.

Is there anything I forgot to add?