search

eriklindernoren / PyTorch-GAN

PyTorch implementations of Generative Adversarial Networks.
MIT License
16.22k stars 4.05k forks source link

Using torch.nn.CrossEntropyLoss along with torch.nn.Softmax output layer ? #150

Open BenoitLeguay opened 3 years ago

BenoitLeguay commented 3 years ago

https://github.com/eriklindernoren/PyTorch-GAN/blob/36d3c77e5ff20ebe0aeefd322326a134a279b93e/implementations/acgan/acgan.py#L113

As stated here: https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html

The CrossEntropyLoss from pytorch combines a LogSoftmax and a NLLLoss. Since you already have a Softmax layer as output activation function for your auxiliary prediction, shouldn't you be using the Negative log likelihood (NLLLoss) as auxiliary loss function ?

I might not get the point here but in any case I am curious to understand the reason behind this choice.

https://stackoverflow.com/questions/55675345/should-i-use-softmax-as-output-when-using-cross-entropy-loss-in-pytorch

lusc1996 commented 3 years ago

I agree with you, the CrossEntropyLoss Function has implemented the SoftMax