search

jr-robotics / robo-gym

An open source toolkit for Distributed Deep Reinforcement Learning on real and simulated robots.
https://sites.google.com/view/robo-gym
MIT License
426 stars 75 forks source link

Choice of state vector for the "EndEffectorPositioningURSim-v0" env #34

Closed brunoabpereira closed 3 years ago

brunoabpereira commented 3 years ago

Hey, great job on this framework!

I've been playing with different DRL algorithms on the "EndEffectorPositioningURSim-v0" env and I've managed to make it work. I have two questions regarding your choice of the state vector:

  1. why use the target coordinates with respect to the end effector reference frame rather than to the simulation reference frame at the base of the robot? (is it related with visual servoing?)
  2. why only normalize the joint_positions and not the entire state? Could normalizing the joint_positions and target_polar yield better results in training?

Thanks, Bruno

matteolucchi commented 3 years ago

Hi @brunoabpereira! Thank you for your kind words!

why use the target coordinates with respect to the end effector reference frame rather than to the simulation reference frame at the base of the robot? (is it related with visual servoing?)

We found out that this makes the problem way easier to solve for the learning agent, and given that the transformation between end effector and robot base is known and can be calculated at any time we see no point in not providing this information to the agent.

why only normalize the joint_positions and not the entire state? Could normalizing the joint_positions and target_polar yield better results in training?

Unfortunately we did not have time to test the performance of this combination, I cannot say whether this would help or not but it would be definitely interesting to try that! If you get the chance, and try this, I would be happy if you can report back the results that you obtain!

I hope this answers your questions. If you any further questions don't hesitate to ask.

Cheers, Matteo