minor changes

[draix]

Hi,

congrats for your great work!

I'm following the tutorial. Got these two errors:

• ValueError: could not broadcast input array from shape (1000) into shape (4096) --> downsized matrix
• RuntimeError: Language not supported: en_glove_cc_300_1m_vectors --> used spacy ver 0.100.6

Did a couple of fixes below.

Still fighting with:

Loading image features ... Loading question features ... Loading VQA Model ... Traceback (most recent call last): File "demo.py", line 110, in main() File "demo.py", line 94, in main vqa_model = get_VQA_model(VQA_model_file_name, VQA_weights_file_name) File "demo.py", line 59, in get_VQA_model vqa_model = model_from_json(open(VQA_model_file_name).read()) File "/Users/draix/Dropbox/Dev/Envs/vqa/lib/python2.7/site-packages/keras/models.py", line 33, in model_from_json return layer_from_config(config, custom_objects=custom_objects) File "/Users/draix/Dropbox/Dev/Envs/vqa/lib/python2.7/site-packages/keras/utils/layer_utils.py", line 24, in layer_from_config class_name = config['class_name'] KeyError: 'class_name'

-- Will update tomorrow.

[iamaaditya]

@draix

Current VQA Demo works only with Keras 0.3

The new Keras (1.0+) does not have pop layer and thus the last layer of the VGG net is not removed. If you see the model of the VGG Net the final layer is the output of the softmax (1000 categories of Imagenet Challenge)

If you change the dimension to 1000 (from 4096) you are effectively using the last layer output and thus you are identifying only one "object" (among the 1000 imagenet classes) and trying to use that as feature for question answering. This will not work, as the questions are not always about the main object of the image.

I have been slacking (due to other VQA challenge) and have not updated this to Keras 1.0, but I plan to do so very soon (maybe a day or two). However, if you want to work with current version I am afraid you will have to use older version of Keras.

2. I will have to look at the change about the Spacy, as when I initially made this adding glove vector was still in beta.

[draix]

Thanks for your prompt reply, and congrats again for your great work.

Got it working now.

Added a requirements.txt file with the dependencies installed and package versions supported on my running version, and updated the Readme notes, in case you find it interesting. I've struggled a lot with the spacy error. It seems is a bug on the new versions.

Best, M

[iamaaditya]

@draix Please use the new code I have pushed. This properly does the layer removal and maintains the 4096 features. (If you use 1000 features like the code you pushed, you will not get good results). Also I have uploaded the weights trained on Keras 1.0. (There are issues on loading weights of previous layers because Keras 1.0 counts layers differently than 0.3)