shreyans29 / thesemicolon

This repository contains Ipython notebooks and datasets for the data analytics youtube tutorials on The Semicolon.
https://www.youtube.com/c/thesemicolon
389 stars 435 forks source link

Accuracy very low and not improving #16

Open bhawmik opened 6 years ago

bhawmik commented 6 years ago

Hi, I am using your basic LSTM architecture to recreate the chatbot. However, I am using GloVe embedding. During my training process, my Training accuracy gets stuck at very low values (0.1969) and no progress happens. I am attaching my code below. Can you tell me what can be done to improve the training?

from keras.models import Sequential from keras.layers import Embedding, Flatten, Dense, LSTM from keras.optimizers import Adam

model.reset_states()

model=Sequential() model.add(Embedding(max_words,embedding_dim,input_length=maxlen)) model.add(LSTM(units=100,return_sequences=True, kernel_initializer="glorot_normal", recurrent_initializer="glorot_normal", activation='sigmoid')) model.add(LSTM(units=100,return_sequences=True, kernel_initializer="glorot_normal", recurrent_initializer="glorot_normal", activation='sigmoid')) model.add(LSTM(units=100,return_sequences=True, kernel_initializer="glorot_normal", recurrent_initializer="glorot_normal", activation='sigmoid')) model.add(LSTM(units=100,return_sequences=True, kernel_initializer="glorot_normal", recurrent_initializer="glorot_normal", activation='sigmoid')) model.summary()

model.layers[0].set_weights([embedding_matrix]) model.layers[0].trainable = False

model.compile(loss='cosine_proximity', optimizer='adam', metrics=['accuracy'])

model.compile(loss='mse', optimizer='adam', metrics=['accuracy'])

model.fit(x_train, y_train, epochs = 500, batch_size = 32, validation_data=(x_val,y_val))

Epoch 498/500 60/60 [==============================] - 0s 3ms/step - loss: -0.1303 - acc: 0.1969 - val_loss: -0.1785 - val_acc: 0.2909 Epoch 499/500 60/60 [==============================] - 0s 3ms/step - loss: -0.1303 - acc: 0.1969 - val_loss: -0.1785 - val_acc: 0.2909 Epoch 500/500 60/60 [==============================] - 0s 3ms/step - loss: -0.1303 - acc: 0.1969 - val_loss: -0.1785 - val_acc: 0.2909

Further training (on the same conversation data set ) does not improve accuracy.