how to forward a sequencer module step by step? for sampling

[Tae618]

I have built a model as follows

function layer:buildLanguageModel()
  local dec = nn.Sequential()
  dec:add(nn.LookupTableMaskZero(self.vocabsize+1, self.hiddensize))
  dec.lstmLayers = {}
  for i=1, self.nlayer do  
      dec.lstmLayers[i] = nn.LSTM(self.hiddensize, self.hiddensize):maskZero(1)
      dec:add(nn.Sequencer(dec.lstmLayers[i]))
  end
  dec:add(nn.Sequencer(nn.MaskZero(nn.Linear(self.hiddensize, self.vocabsize+1), 1)))
  dec:add(nn.Sequencer(nn.MaskZero(nn.LogSoftMax(), 1)))

  return dec
end

In the training phase, it will use a sequence of words as input. But in the evaluating phase, I use a start token as input for t=0 step forward and then the input of t step will be the output of t-1 step. So I need forward the sequencer module step by step from 1 to sequence length step. Anyone knows how to do this ?

[Tae618]

I wonder if sequencer module can forward at single time-step. Maybe it must forward for sequence and can't forward a single step.

[JoostvDoorn]

You can use remember with Sequencer. You might find it helpful to have a look at #304 .

[Tae618]

Thank you for your suggestion, I have known how to deal with it. Just like #247 does and making some changes will be ok.

[sudongqi]

The solution on #247 doesn't work for me. Maybe it's because I am using SeqLSTM and SeqLSTM handle forget and remember differently? I will do a testing on an overfitted example.

Update: This bug is caused by not loading the model properly. #353