search

salaniz / pytorch-gve-lrcn

PyTorch implementations for "Generating Visual Explanations" (GVE) and "Long-term Recurrent Convolutional Networks" (LRCN)
MIT License
92 stars 22 forks source link

Class embeddings to GVE ? #6

Open saurabhsharma1993 opened 6 years ago

saurabhsharma1993 commented 6 years ago

Hi, In the GVE paper, the LRCN is modified so that class embeddings are passed at every time step to the second LSTM. I see that you are appending the one-hot class labels to the image features. This is different from the paper, which uses class embeddings comptued from average hidden state for a language model trained on the image features. You should correct this.

salaniz commented 6 years ago

It works well enough with one-hot class embeddings. If you want this implementation to learn/support LSTM class embeddings, feel free to create a pull request.