search

vijayvee / video-captioning

This repository contains the code for a video captioning system inspired by Sequence to Sequence -- Video to Text. This system takes as input a video and generates a caption in English describing the video.
MIT License
165 stars 67 forks source link

not the output in demo #17

Closed Aishwaryabaskaran closed 4 years ago

Aishwaryabaskaran commented 5 years ago

Hello, First of all, thanks for the repo. It's of great help. I tried using it with the pretrained model ckpt provided. The outputs are gibberish and not any useful sentence. I am unable to figure out what the issue is. Please help.

rupeshchandgude commented 5 years ago

I am facing same problem. I have followed the sequence mentioned and trained the model. while training the results shown are relatable with GT. But while running test file, it always shows same sentence for every video and the sentence contains same words repeatably. I have attached the error file which contains the training results and testing results.

error.txt

vijayvee commented 5 years ago

Hi @rupeshchandgude, I think the error that you're mentioning is slightly different from what @Aishwaryabaskaran was facing since your predictions are not using the same model that I provided with the repo. Could you please check if your results make more sense when restoring from the checkpoint I provided?

From your output, it seems that your model is overfitting on the training set. Are you sure you are using regularization techniques correctly in the model?

abhaygargab commented 5 years ago

Was this problem solved?? I am using the model weights and checkpoint from the repository but my output is always same (repeated words without ) for any input. Will be thankful for any hints on this.

vijayvee commented 5 years ago

Thanks for your interest in the repo, I don't think this is a wrong checkpoint file since I created my demo video with this file. How does the train set output look for you?

abhaygargab commented 4 years ago

I solved the issue by replacing the line in Extract_Feats.py- frame = skimage.transform.resize(frame,[224,224]) with frame = skimage.transform.resize(frame,[224,224], preserve_range=True). I found this solution following this issue- https://github.com/chenxinpeng/S2VT/issues/5

vijayvee commented 4 years ago

That's great, thanks for sharing @abhaygargab ! Closing this issue for now, please reopen if the above solution doesn't work for anyone.