Fix padding issues with RNN model

Michael0x2a / nlp-capstone

1 stars 0 forks source link

Fix padding issues with RNN model #2

Open Michael0x2a opened 7 years ago

Michael0x2a commented 7 years ago

Ideas to explore:

[ ] Apply a mask before computing loss
[x] Look into using dynamically-resizing RNNs, and having the width vary per batch

Potential issues:

[ ] Comment width can vary widely; how do we handle outliers?

Michael0x2a commented 7 years ago

Well, I attempted looking into using dynamically-resizing RNNs, and feeding in sequence length, and other such things, and I got basically the same results, except that each epoch took 3 times longer to run...

I'm a bit stumped as to how I'd apply a mask before computing loss, given that I'm comparing the softmax cross entropy between the output labels (and so a mask isn't applicable).