second layer of static word embeddings

bernhard2202 / twitter-sentiment-analysis

Twitter Sentiment Analysis with Deep Convolutional Neural Networks and LSTMs in TensorFlow.

38 stars 9 forks source link

Closed bernhard2202 closed 8 years ago

bernhard2202 commented 8 years ago

Use two layers for the word embeddings - static and non static.

Report on results.

bernhard2202 commented 8 years ago

Experiment results

Yes, there is an improvement but it is not significant.
After 40k training steps the validation accuracy was at 0.858 with static layer and 0.856 without.
Training itself is far more slow with the second layer.
This method might improve after implementing #7

bernhard2202 commented 8 years ago

In the end no static version could beat the corresponding non-static version on Kaggle - therefore closed.