how to implement batch_chat for videochat2 benchmark?

OpenGVLab / Ask-Anything

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

https://vchat.opengvlab.com/

MIT License

2.85k stars 230 forks source link

how to implement batch_chat for videochat2 benchmark? #187

Open dragen1860 opened 3 weeks ago

dragen1860 commented 3 weeks ago

Dear author: Thnaks for the really solid work of videochat2. Recently i finetunning a individual model on my custom dataset and want to test on my benchmark samples, which as up to 30K samples. It is extreme time-consuming for chat one by one and then compute the accuracy. Does anyone know how to implement a batch_chat like sytle parallel inference function, so that i could inference several samples meanwhile. thanks.

Andy1621 commented 3 weeks ago

Hi! We will release the vllm verision for batch_chat recently.