search

OpenGVLab / InternVL

[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型
https://internvl.readthedocs.io/en/latest/
MIT License
5.24k stars 409 forks source link

Streaming output support? #160

Closed xiangqi1997 closed 4 weeks ago

xiangqi1997 commented 3 months ago

Wondering if streaming output is supported? Or are there any results about the time to first token and time per output token? Thanks.

NiYueLiuFeng commented 3 months ago

Copying the code of def stream_chat(self, ....) to modeling_internvl_chat.py from modeling_internlm2.py https://huggingface.co/OpenGVLab/InternVL-Chat-V1-5/blob/main/modeling_internlm2.py and make very small changes , I implement it and verify that is usefull.

LIMr1209 commented 2 months ago

@NiYueLiuFeng Can you share your modeling_internvl_chat.py？ Thank you very much

czczup commented 4 weeks ago

Hi, see this guide for streaming output: https://internvl.readthedocs.io/en/latest/internvl2.0/quick_start.html#streaming-output