Possible PyTorch version error

[windweller]

Hi,

I'm using PyTorch 0.2.0 I trained a model using this code's default setting already, and just want to reload it and evaluate it on validation and test dataset.

However, I encountered this error:

togrep : ['--nlipath', 'dataset/SNLI/', '--n_epochs', '1', '--gpu_id', '2']

Namespace(batch_size=64, decay=0.99, dpout_fc=0.0, dpout_model=0.0, enc_lstm_dim=2048, encoder_type='BLSTMEncoder', fc_dim=512, gpu_id=2, lrshrink=5, max_norm=5.0, minlr=1e-05, n_classes=3, n_enc_layers=1, n_epochs=1, nlipath='dataset/SNLI/', nonlinear_fc=0, optimizer='sgd,lr=0.1', outputdir='savedir/', outputmodelname='model.pickle', pool_type='max', seed=1234)
** TRAIN DATA : Found 942069 pairs of train sentences.
** DEV DATA : Found 19657 pairs of dev sentences.
** TEST DATA : Found 19656 pairs of test sentences.
Found 102367(/123296) words with glove vectors
Vocab size : 102367
NLINet (
  (encoder): BLSTMEncoder (
    (enc_lstm): LSTM(300, 2048, bidirectional=True)
  )
  (classifier): Sequential (
    (0): Linear (16384 -> 512)
    (1): Linear (512 -> 512)
    (2): Linear (512 -> 3)
  )
)

TEST : Epoch 1
Traceback (most recent call last):
  File "train_nli.py", line 300, in <module>
    evaluate(0, 'test', True)
  File "train_nli.py", line 248, in evaluate
    output = nli_net((s1_batch, s1_len), (s2_batch, s2_len))
  File "/home/anie/miniconda2/envs/tensorflow/lib/python2.7/site-packages/torch/nn/modules/module.py", line 224, in __call__
    result = self.forward(*input, **kwargs)
TypeError: forward() takes exactly 2 arguments (3 given)

Is this a PyTorch version-related error? I didn't change any code except commenting out the training part.

[aconneau]

I think you saved only the encoder. And so you loaded the encoder instead of the full model, and then you are trying to forward the ((s1_batch, s1_len), (s2_batch, s2_len)) tuple to the "forward" function of the encoder, while it is meant to be fed to the "forward" function of the full model.

Since InferSent is meant to use only the sentence encoder, I did not save the classifier at the end of train_nli.py ( https://github.com/facebookresearch/InferSent/blob/master/train_nli.py#L303 ) . I will make a slight modification so that it saves both.

[windweller]

Can I just change it to:

torch.save(nli_net,
           os.path.join(params.outputdir, params.outputmodelname))

?

[aconneau]

https://github.com/facebookresearch/InferSent/commit/bcc794990d40774698ab492d29396721ae0b2f9c should fix it. Then you load either: os.path.join(params.outputdir, params.outputmodelname) : full model (encoder + classifier) os.path.join(params.outputdir, params.outputmodelname + '.encoder')) : just the encoder

[windweller]

Cool, thanks :)