Closed lihuiliullh closed 5 years ago
Hi.
It is an implementation of the common softmax cross-entropy loss function.
However, because of the output are already probabilities (softmax), we cannot use the xent = tf.nn.softmax_cross_entropy_with_logits(logits, labels)
method of Tensorflow.
We reimplemented it, avoiding numerical instability with the first line: (output = tf.maximum(output, self.EPSILON, name="Avoiding_explosions")
- it avoids to have zero values in the output and hence compute the softmax with them, resulting in Nan values.
Here you can find a reference for the cross-entropy reimplementation
Dear Authors:
May I know what is the meaning of the loss function in Net_Subgraph? It is a little difficult for me to understand.
output = tf.maximum(output, self.EPSILON, name="Avoiding_explosions") # to avoid explosions xent = -tf.reduce_sum(target * tf.log(output), 1) lo = tf.reduce_mean(xent)
Best.