search

awjuliani / DeepRL-Agents

A set of Deep Reinforcement Learning Agents implemented in Tensorflow.
MIT License
2.23k stars 825 forks source link

Solving NaN vlaues #34

Open IbrahimSobh opened 7 years ago

IbrahimSobh commented 7 years ago

Dears Dear @DMTSource

Sometimes NaN values appear and agent just collapses. I used the following code to solve this issue.

After policy ... 

           self.policy = slim.fully_connected(rnn_out,a_size,
                activation_fn=tf.nn.softmax,
                weights_initializer=normalized_columns_initializer(0.01),
                biases_initializer=None)

just add this line (to prevent log(0) )

self.policy += 1e-7

This line should protect the following code ... 

self.entropy = - tf.reduce_sum(self.policy * tf.log(self.policy))
self.policy_loss = -tf.reduce_sum(tf.log(self.responsible_outputs)*self.advantages)

If you have any comments and suggestions please share ...