search

xbpeng / DeepTerrainRL

terrain-adaptive locomotion skills using deep reinforcement learning
GNU Lesser General Public License v3.0
435 stars 129 forks source link

What should I do if I want to train a DDPG model? #30

Open YurongYou opened 7 years ago

YurongYou commented 7 years ago

Hi, I am currently doing a project about trying to use DDPG to train the same agent in this paper. It seems that I need to change the network architecture and the way to update the network a little bit. I have tried diving into the source code for two days, but there are just too many classes and I still have not found what I should modify. Could you give me some advice or explain a little bit which class is doing what in training? (e.g. cScenarioTrain, cScenarioExp, cNeuralNetLearner, etc.) Thanks a lot!

xbpeng commented 7 years ago

I think implementing DDPG will be pretty difficult. But a good start might be to look into cNeuralNetTrainer. Trainers are the set of classes responsible for updating the networks. ScenarioExp are the classes responsible for generating the training data, and ScenarioTrain manages a bunch of ScenarioExps to generate data in parallel to feed into the trainer. A good example to follow might be to look at ScenarioTrainCacla, which uses an actor-critic framework. But again, it will be pretty tricky to get DDPG setup.

YurongYou commented 7 years ago

Thank you very much! This would help a lot~