search

torch / nn

Other
1.34k stars 967 forks source link

SoftMax and LogSoftMax behave differently with multidimensional input #548

Open dpfau opened 8 years ago

dpfau commented 8 years ago

SoftMax will treat 4D input as a minibatch and sum along the second dimension, while LogSoftMax will throw an error, claiming it was expecting a vector or matrix. It seems like the two should behave similarly.

fmassa commented 8 years ago

They used to behave similarly before commit https://github.com/torch/nn/pull/370 , which extended SoftMax to work in the spatial domain. LogSoftMax could be extended similarly as well, but no one needed it.

fmassa commented 8 years ago

I was wrong when I said that no one needed SpatialLogSoftMax. It's present in cudnn actually. I'll prepare a PR extending LogSoftMax to work in the spatial domain