search

araffin / robotics-rl-srl

S-RL Toolbox: Reinforcement Learning (RL) and State Representation Learning (SRL) for Robotics
https://s-rl-toolbox.readthedocs.io
MIT License
607 stars 91 forks source link

why zmp is used? #52

Closed huiwenzhang closed 5 years ago

huiwenzhang commented 5 years ago

Hi,

To retrieve encoded states from SRL module, the zmp pakcage and a fashion of client and sever socket communication are leveraged. My question is why not just load the pretrained encoder, feed the raw observations and get state representation.

Thanks

araffin commented 5 years ago

Hello,

You are maybe talking about that part of the code: https://github.com/araffin/robotics-rl-srl/blob/master/environments/srl_env.py#L33 ?

the zmp pakcage and a fashion of client and sever socket communication are leveraged.

zmq is only used with real robots to interface with ROS which only support python 2.

My question is why not just load the pretrained encoder, feed the raw observations and get state representation.

However, we are using multiprocessing and queues (see here) for using the SRL model. This is for performance reason. When using multiprocessing with a RL algo, we had to duplicate the SRL model (which takes a lot of RAM and GPU memory). It is more efficient to have only one instance of the SRL model that get pinged (it receives an observation and sends out the state) by the different environments that run in parallel.

huiwenzhang commented 5 years ago

Get it. Thanks