I have variable length sequences and I am zero padding them and adding them to a batch. I am using LSTM decorated with a Sequencer to classify these sequences. I have two questions:
2) Does MaskZero skip all the zeros in the input while forward and backward or it only skips those zeros which are continuous and till the end of tensor. How does MakeZero differentiate whether zero in the input row is actual data or padding?
For ex. A sequence can look like this: 4, 5, 0, 5, 0, 0, 0
Will MaskZero process the zero at third location correctly?
Hi,
I have variable length sequences and I am zero padding them and adding them to a batch. I am using
LSTMdecorated with aSequencerto classify these sequences. I have two questions:1) Does it matter if I zero pad smaller sequences on LEFT or RIGHT? Following discussion https://github.com/Element-Research/rnn/issues/75 suggests left padding should be done. Is this correct?
2) Does
MaskZeroskip all the zeros in the input while forward and backward or it only skips those zeros which are continuous and till the end of tensor. How doesMakeZerodifferentiate whether zero in the input row is actual data or padding?For ex. A sequence can look like this: 4, 5, 0, 5, 0, 0, 0
Will
MaskZeroprocess the zero at third location correctly?