Line 182 in `caption_generator.py` of `im2txt` should be `[-self.beam_size:]` instead of `[:-self.beam_size]` since the comment above explicitly says the "last `beam_size` elements"

ZisIsNotZis commented 5 years ago

System information

What is the top-level directory of the model you are using: research/im2txt
Have I written custom code (as opposed to using a stock example script provided in TensorFlow): no
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
TensorFlow installed from (source or binary): binary, anaconda
TensorFlow version (use command below): b'unknown' 1.12.0
Bazel version (if compiling from source): 0.22.0- (@non-git)
CUDA/cuDNN version: 10.1
GPU model and memory: GeForce GTX 1080 / 8116MiB
Exact command to reproduce: N/A

Describe the problem

Line 182 in caption_generator.py of im2txt should be [-self.beam_size:] instead of [:-self.beam_size], since the comment above explicitly says the "last beam_size elements".

    # For this partial caption, get the beam_size most probable next words.
    # Sort the indexes with numpy, select the last self.beam_size
    # (3 by default) (ie, the most likely) and then reverse the sorted
    # indexes with [::-1] to sort them from higher to lower.
    most_likely_words = np.argsort(word_probabilities)[:-self.beam_size][::-1]

run_inference will output length 1 description without this, like

Captions for image COCO_val2014_000000224477.jpg:
  0)  (p=0.000000)
  1) surfer (p=0.000000)
  2) man (p=0.000000)

while it will output much more "normal" output with this:

Captions for image COCO_val2014_000000224477.jpg:
  0) a person riding a surf board on a wave <S> <S> <S> <S> <S> <S> <S> <S> . (p=0.000896)
  1) a person riding a surf board on a wave <S> <S> <S> <S> <S> <S> . <S> <S> (p=0.000732)
  2) a person riding a surf board on a wave <S> <S> <S> <S> <S> . <S> <S> <S> (p=0.000456)

Source code / logs

N/A

xieyxclack commented 5 years ago

Thank you very much!! It works

senorpinatta commented 5 years ago

When I change line 182 to: most_likely_words = np.argsort(word_probabilities)[-self.beam_size][::-1]

I get: IndexError: invalid index to scalar variable.

Did anyone face this or know how to solve it?

Edit

I changed line 182 to: most_likely_words = np.argsort(word_probabilities)[::-1] and that worked for me.

fengyang0317 commented 5 years ago

Please fix this @cshallue. Thank you.

tensorflowbutler commented 4 years ago

Hi There, We are checking to see if you still need help on this, as this seems to be an old issue. Please update this issue with the latest information, code snippet to reproduce your issue and error you are seeing. If we don't hear from you in the next 7 days, this issue will be closed automatically. If you don't need help on this issue any more, please consider closing this.

fengyang0317 commented 4 years ago

The problem is not fixed yet.

tensorflow / models