why it didn't use softmax in computing multichoice loss

huggingface / pytorch-openai-transformer-lm

🐥A PyTorch implementation of OpenAI's finetuned transformer language model with a script to import the weights pre-trained by OpenAI

MIT License

1.51k stars 285 forks source link

Closed eveliao closed 5 years ago

eveliao commented 5 years ago

the input are just logits, not normalized by softmax, why can we directly compute the cross entropy loss with it and y?

eveliao commented 5 years ago

cross entropy includes log_softmax function, sorry...