search

modelscope / evalscope

A streamlined and customizable framework for efficient large model evaluation and performance benchmarking
https://evalscope.readthedocs.io/en/latest/
Apache License 2.0
263 stars 33 forks source link

evalscope 5.5 stream.日志不起作用 #183

Open jinweida opened 2 weeks ago

jinweida commented 2 weeks ago

evalscope perf --url 'http://127.0.0.1:9600/v1/chat/completions' \ --parallel 128 --model 'Qwen2_5-32B-Instruct' --log-every-n-query 10 --read-timeout=360 \ --dataset-path '/home/ai/.cache/evalscope/openqa.jsonl' -n 2048 --max-prompt-length 128000 \ --tokenizer-path '/home/ai/hub/Qwen25-32B-Instruct' \ --api openai --stop '<|im_end|>' --stream --dataset openqa

使用者命令

image

一致在输出流式的结果。5.2 是可以的

Yunnglin commented 2 weeks ago

目前在修改这部分逻辑，临时使用的话请注释掉下面一行： https://github.com/modelscope/evalscope/blob/671e71bd214eba1c7e9fa1f90e0f0d4ff9ab04e5/evalscope/perf/http_client.py#L570