search

kelvinxu / arctic-captions

960 stars 349 forks source link

Feature extraction #6

Closed hyunseokmin closed 9 years ago

hyunseokmin commented 9 years ago

Hello, thank you for sharing this great project.

I would like to run the code for caption generation of my own image. To that end, I tried to extract image feature vector with VGG model (based on the paper). I used below code for extract feature vector (values of conv5_3).

input_img = numpy.array( caffe.io.load_image(path_img) ) #loading images HxWx3 (RGB) caffe_input = numpy.array( preprocess_image(input_img) ) #preprocess the images

caffe_net.blobs['data'].reshape(1,3,224,224) caffe_net.blobs['data'].data[...] = caffe_input

out = caffe_net.forward()

feature = numpy.array(caffe_net.blobs['conv5_3'].reshape(1,512,14,14))

How can I get context vector for function 'capgen.gen_sample'? I am not good at python and caffe. I want to know how to get 'context' from the VGG model.

Thank you.

kelvinxu commented 9 years ago

hey hyunseok, sorry for the delayed response. In caffe, after you use the forward method, you can recover the activations by accessing the caffe_net.blobs dictionary. One thing to remember is that you'll need to reshape the image and center crop as we described in the paper. Hope that helps.

hyunseokmin commented 9 years ago

Thank you. I can extract image feature using VGG 19 layer model, (conv5_4 layer).