search

sgrvinod / a-PyTorch-Tutorial-to-Image-Captioning

Show, Attend, and Tell | a PyTorch Tutorial to Image Captioning
MIT License
2.72k stars 711 forks source link

Run captioning to my own images. #88

Open gxskar opened 4 years ago

gxskar commented 4 years ago

Sorry for the stupid question, but I am very new to machine learning applications. I have run your code until the point of having a checkpoint. I now want to produce a list of captions from some images that I have gathered. What are the steps for doing so? Again sorry, and thanks.

alexandre-blanc commented 4 years ago

I have been working on this tutorial for the past two weeks. As far as I understand, the forward method of the decoder cannot be used for inference, since it makes use of the technique called "teacher forcing". Basically, this means that to evaluate the model, you have to feed it not only an image, but also a caption !

This seem to defeat the purpose of modeling, because we would need to have a proper caption to generate one. This issue is actually adressed in the remarks : here

kmario23 commented 4 years ago

Sorry for the stupid question, but I am very new to machine learning applications. I have run your code until the point of having a checkpoint. I now want to produce a list of captions from some images that I have gathered. What are the steps for doing so? Again sorry, and thanks.

Do you already have a trained model? If yes, then simply running caption.py on your images should work fine. This is because we're using beam search in the evaluation mode (i.e. it doesn't need a caption for the image. Just the images should be sufficient)

Mollylulu commented 4 years ago

Sorry for the stupid question, but I am very new to machine learning applications. I have run your code until the point of having a checkpoint. I now want to produce a list of captions from some images that I have gathered. What are the steps for doing so? Again sorry, and thanks.

Do you already have a trained model? If yes, then simply running caption.py on your images should work fine. This is because we're using beam search in the evaluation mode (i.e. it doesn't need a caption for the image. Just the images should be sufficient)

Sorry, When I run caption.py, there is an ERROR, the decoder has no attribute decode_step, how to handle this issue?

alexandre-blanc commented 4 years ago

Could you show us the exact commands you used to :

  1. obtain the checkpoint
  2. run caption.py

as well as the full output in the terminal (i.e. the complete error message) ?

Mollylulu commented 4 years ago

Warning should be okay for running the code, which is caused by the Scipy version.

Best Regards Wanlu Luo

发件人: AnukritiSinghh 发送时间: 2020年7月31日 2:59 收件人: sgrvinod/a-PyTorch-Tutorial-to-Image-Captioning 抄送: Mollylulu; Comment 主题: Re: [sgrvinod/a-PyTorch-Tutorial-to-Image-Captioning] Run captioning to my own images. (#88)

I get the error caption.py:33: DeprecationWarning: imreadis deprecated!imreadis deprecated in SciPy 1.0.0, and will be removed in 1.2.0. Use imageio.imread instead. img = imread(image_path) caption.py:37: DeprecationWarning:imresizeis deprecated!imresize is deprecated in SciPy 1.0.0, and will be removed in 1.3.0. Use Pillow instead: numpy.array(Image.fromarray(arr).resize()). img = imresize(img,(256, 256)) and The default multichannel argument (None) is deprecated. Please specify either True or False explicitly. multichannel will default to False starting with release 0.16. warn('The default multichannel argument (None) is deprecated. Please ' — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.