search

avisingh599 / visual-qa

[Reimplementation Antol et al 2015] Keras-based LSTM/CNN models for Visual Question Answering
https://avisingh599.github.io/deeplearning/visual-qa/
MIT License
481 stars 186 forks source link

Possible Bug: not handling question mark #11

Closed dhruvbatra closed 8 years ago

dhruvbatra commented 8 years ago

It's possible I'm mistaken, but it seems there's a bug in the way word embeddings are being computed in own_image.py

       question = unicode(raw_input("Ask a question: "))
       X_q = get_questions_tensor_timeseries([question], nlp, timesteps)

If question = "what color is the cat?"

the word "cat?" will considered out of vocabulary (due to lack of space before question mark) and word embeddings will be an all zero vector.

avisingh599 commented 8 years ago

spaCy, the library I used for the NLP part, takes care of the lack of space. Its tokenization takes care of questions marks, apostrophes, lowering the case etc.

dhruvbatra commented 8 years ago

Hmm. True. Then there's some else going on because I am seeing the last vector as all zero.

dhruvbatra commented 8 years ago

Closing for now.

avisingh599 commented 8 years ago

The last all zeros corresponds to the question mark.

dhruvbatra commented 8 years ago

Got it. It feels like an odd representation of that symbol, but thanks.