Open yangyuya opened 2 months ago
Hi so you want to run the demo without ASR and even without loading whisper model?
Please change the demo.yaml config to disable ASR, https://github.com/bytedance/Shot2Story/blob/7ddced29ff8d639e52543bf13e801f3ba0b26d46/code/lavis/projects/blip2/eval/demo.yaml#L20 to False
and https://github.com/bytedance/Shot2Story/blob/5bc6cf8d351a407230ab82c961ba3834526c2bdd/code/lavis/projects/blip2/eval/demo.yaml#L27 to Describe this video in detail.
Hi so you want to run the demo without ASR and even without loading whisper model?
Yes, because the videos in activitynet caption don't have ASR, and if I load whisper model I get oom on a 4090 24G GPU.
I have already change the demo.yaml config to disable ASR, then I still get the same problem above.
@yangyuya Hi have you solved this issue on 4090 GPU? Would you mind providing more details of you environment, errors? I fail to find a 4090 GPU to test.
Thanks for your job!
I don't load whisper and set all ars empty, then I try to run demo_video.py just using upload_vid as follows:
But I failed: 1、I get error:
2、If I set do_sample=False in "self.model.llama_model.generate",the error above can be solved. But the "summary" will be:
I can't solve this problem, could you help me test my own video without whisper?