search

IntelLabs / coach

Reinforcement Learning Coach by Intel AI Lab enables easy experimentation with state of the art Reinforcement Learning algorithms
https://intellabs.github.io/coach/
Apache License 2.0
2.32k stars 459 forks source link

The network's configuration of CARLA_DDPG #275

Closed fangchuan closed 5 years ago

fangchuan commented 5 years ago

Hi，I try to run The CARLA_DDPG preset in coach，with The help of The document，I run it successfully. However，i want to dig into the implementation of DDPG in coach, I have reviewed The code of CARLA_DDPG.py, and i have figured out both the network of actor and critic, as the below picture, who can help me revise the understanding and give me some supplementary advise?

fangchuan commented 5 years ago

ddpg_actor_critic_network

huang-CX commented 5 years ago

Hi, it is hard to be convergence if the network parameters are randomly initialized. I have tried to initialize the convolutional layers with pretrained parameters, and it performed better.

galnov commented 5 years ago

@fangchuan you can use the --print_networks_summary flag to print the networks' structure.

fangchuan commented 5 years ago

@huang-CX ,where did you find the pretrained parameter?

fangchuan commented 5 years ago

hi, I have a question when I print the network's summary: although there are batch_normalization layers inserted between each convolutional layers or dense layers in the network, it does not set the is_training flag to True. It means the network didn't use batch_normalization at all, why you did not use batch_normalization in DDPG? I'm looking forward to your answer, @galnov

fangchuan commented 5 years ago

Another question in CARLA_DDPG is about the activation_fn used in critic_network, whether the critic_network use the relu activation function in output head(v_head)? Because when i use the relu in critic_network's output layer in my ddpg implementation, I found it was stuck in the dead area at the beginning of training.

HubFire commented 5 years ago

Hi，I try to run The CARLA_DDPG preset in coach，with The help of The document，I run it successfully. However，i want to dig into the implementation of DDPG in coach, I have reviewed The code of CARLA_DDPG.py, and i have figured out both the network of actor and critic, as the below picture, who can help me revise the understanding and give me some supplementary advise?

Hi ,When I run CARLA DDPG ,after 10 episodes ,the car doesn't move . is there a trained model to test

huang-CX commented 5 years ago

@fangchuan you can use a pretrained VGG to extract feature from raw image.

galnov commented 5 years ago

hi, I have a question when I print the network's summary: although there are batch_normalization layers inserted between each convolutional layers or dense layers in the network, it does not set the is_training flag to True. It means the network didn't use batch_normalization at all, why you did not use batch_normalization in DDPG? I'm looking forward to your answer, @galnov

Batch norm fixed in this PR. Note that it is still disabled in DDPG as results were better without it.

galnov commented 5 years ago

Closing as the questions have been answered. If this is still not resolved, please reopen.