search

LLaVA-VL / LLaVA-NeXT

Apache License 2.0
2.83k stars 229 forks source link

video_demo.py code didn't work #111

Closed MSY99 closed 3 months ago

MSY99 commented 3 months ago

I tried to get sample video inference result by using the command below, but it didn't work.

bash scripts/video/demo/video_demo.sh lmms-lab/LLaVA-NeXT-Video-7B-DPO vicuna_v1 32 2 True data/sample.mp4

I got this error msg... but I couldn't fix it

Anybody got this error?

ZhangYuanhan-AI commented 3 months ago

Hi, it should be :bash scripts/video/demo/video_demo.sh lmms-lab/LLaVA-NeXT-Video-7B-DPO vicuna_v1 32 2 average after no_token True ./data/llava_video/video-chatgpt/evaluation/Test_Videos/v_Lf_7RurLgp0.mp4

MSY99 commented 3 months ago

Hi, it should be :bash scripts/video/demo/video_demo.sh lmms-lab/LLaVA-NeXT-Video-7B-DPO vicuna_v1 32 2 average after no_token True ./data/llava_video/video-chatgpt/evaluation/Test_Videos/v_Lf_7RurLgp0.mp4

Still same error....

ZhangYuanhan-AI commented 3 months ago

try: bash scripts/video/demo/video_demo.sh lmms-lab/LLaVA-NeXT-Video-7B vicuna_v1 32 2 average after no_token True /mnt/bn/vl-research/workspace/yhzhang/LLaVA-NeXT/playground/demo/xU25MMA2N4aVtYay.mp4

gordonhu608 commented 3 months ago

Hi, I can run the demo smoothly, but the response I'm getting is " ". Can you explain what your newly added arguments which are (average after no_token) mean? I assume average is mm_spatial_pool_mode but what about the others? Thanks!

MSY99 commented 3 months ago

try: bash scripts/video/demo/video_demo.sh lmms-lab/LLaVA-NeXT-Video-7B vicuna_v1 32 2 average after no_token True /mnt/bn/vl-research/workspace/yhzhang/LLaVA-NeXT/playground/demo/xU25MMA2N4aVtYay.mp4

I tried it, and also got same error...

this is error msg File "-/LLaVA-NeXT/playground/demo/video_demo.py", line 183, in run_inference(args) File "-/LLaVA-NeXT/playground/demo/video_demo.py", line 160, in run_inference output_ids = model.generate(inputs=input_ids, images=video, attention_mask=attention_masks, modalities="video", do_sample=True, temperature=0.2, max_new_tokens=1024, use_cache=True, stopping_criteria=[stopping_criteria]) UnboundLocalError: local variable 'video' referenced before assignment

ZhangYuanhan-AI commented 3 months ago

Hi, I can run the demo smoothly, but the response I'm getting is " ". Can you explain what your newly added arguments which are (average after no_token) mean? I assume average is mm_spatial_pool_mode but what about the others? Thanks!

try to pull our latest code!

MSY99 commented 3 months ago

When I clone latest code and create new conda virtual env, it works! But I tried to test by using my own data, it still got same error... UnboundLocalError: local variable 'video' referenced before assignment

ZhangYuanhan-AI commented 3 months ago

When I clone latest code and create new conda virtual env, it works! But I tried to test by using my own data, it still got same error... UnboundLocalError: local variable 'video' referenced before assignment

Make sure the path of your video is correct

MSY99 commented 3 months ago

When I clone latest code and create new conda virtual env, it works! But I tried to test by using my own data, it still got same error... UnboundLocalError: local variable 'video' referenced before assignment

Make sure the path of your video is correct

Sorry..! That was my fault. I fixed the path and I could got well Response. Thanks!

ZhangYuanhan-AI commented 3 months ago

Good to know, if it solved, please close this issue, thx.