search

avisingh599 / visual-qa

[Reimplementation Antol et al 2015] Keras-based LSTM/CNN models for Visual Question Answering
https://avisingh599.github.io/deeplearning/visual-qa/
MIT License
480 stars 186 forks source link

Training Loss increases #24

Open abhitrip opened 8 years ago

abhitrip commented 8 years ago

Is the loss being output on trainLSTM_1.py , the real train loss or just the loss of some random epoch ? Cause My training loss seems to increase after 2 epochs ... FYI : I have used the glove vectors as the word vectors.

avisingh599 commented 8 years ago

Does it increase consistently, or only for 1-2 epochs?

abhitrip commented 8 years ago

For the first 4 epochs its like : 215424/215375 [==============================] - 418s - train loss: 3.7656
215424/215375 [==============================] - 432s - train loss: 3.7264
215424/215375 [==============================] - 439s - train loss: 3.7595
215424/215375 [==============================] - 442s - train loss: 3.7989
I have run the code for 20 epochs , and it is continuously increasing

avisingh599 commented 8 years ago

Does not seem right. Try reducing your learning rates, and see what you get. You might also be interested in trying a bunch of different learning rates in parallel, and see which one converges the fastest.

abhitrip commented 8 years ago

I tried reducing the learning rate and yes now the loss decreases generally. Can you give me a fair idea of what was the loss you were getting after around 100 epochs, cause now the losses are decreasing , but not by much ...

avisingh599 commented 8 years ago

I did this project a long time ago, and unfortunately I don't have access to most of the data associated with this project anymore (I graduated undergrad, and do not have access to my machine there anymore).

I remember getting competitive performance on the benchmark around 50-60 epochs, so you might want to try that.

abhitrip commented 8 years ago

Ok , thanks :) let me run a couple more epochs and I will let you know