Is it not supported by streaming output?

[CaoChensy]

Is it not supported by streaming output?

[polanwang404]

demo.py 中的 use_vllm = True # set True to use vllm for inference 就可以了，记得安装 vllm

[CaoChensy]

Thanks!

[leoterry-ulrica]

demo.py 中的 use_vllm = True # set True to use vllm for inference 就可以了，记得安装 vllm

不起作用

[Neo-Zhangjiajie]

Since we need to post-process the LLM output to match the citation number such as [6-8] with the context sentences, it currently does not support stream output.

[leoterry-ulrica]

Since we need to post-process the LLM output to match the citation number such as [6-8] with the context sentences, it currently does not support stream output.

@zRzRzRzRzRzRzR 说的不支持。

[polanwang404]

demo.py 中的 use_vllm = True # set True to use vllm for inference 就可以了，记得安装 vllm

不起作用

But after I set use_vllm to True, it did run, and achieved the similar result as the demo video

[leoterry-ulrica]

demo.py 中的 use_vllm = True # set True to use vllm for inference 就可以了，记得安装 vllm

不起作用

But after I set use_vllm to True, it did run, and achieved the similar result as the demo video

这个跟流式输出无关。