wjko2 / Domain-Agnostic-Sentence-Specificity-Prediction

13 stars 10 forks source link

Default num_layers 1 instead of 3 #13

Open JoshuaMathias opened 2 years ago

JoshuaMathias commented 2 years ago

...\lib\site-packages\torch\nn\modules\rnn.py:62: UserWarning: dropout option adds dropout after all but last recurrent layer, so non-zero dropout expects num_layers greater than 1, but got dropout=0.5 and num_layers=1 warnings.warn("dropout option adds dropout after all but last "

In section 5.2 Training Details of the paper, I see the following quote: "The LSTM encoder generates 100-dimensional representations. For the multilayer perceptron, we use 3 fully connected 100-dimensional layers."

In the code, I see the following parameters right next to each other: parser.add_argument("--enc_lstm_dim", type=int, default=100, help="encoder nhid dimension") parser.add_argument("--n_enc_layers", type=int, default=1, help="encoder num layers")

Solution: I'm planning to change the number of layers to 3 instead of 1.

JoshuaMathias commented 2 years ago

I learned that the parameter n_enc_layers isn't used in models.py. Instead num_layers was hard-coded as 1 every time LSTM or GRU were initialized. I edited every instance I found in models.py to use n_enc_layers instead; then the warning went away. https://github.com/JoshuaMathias/Domain-Agnostic-Sentence-Specificity-Prediction/commit/6303d690b5b5bdd18d284874f6300a8fcd0659c4