modelscope / ms-swift

Use PEFT or Full-parameter to finetune 350+ LLMs or 100+ MLLMs. (LLM: Qwen2.5, Llama3.2, GLM4, Internlm2.5, Yi1.5, Mistral, Baichuan2, DeepSeek, Gemma2, ...; MLLM: Qwen2-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSeek-VL, Phi3.5-Vision, ...)

https://swift.readthedocs.io/zh-cn/latest/Instruction/index.html

Apache License 2.0

4.01k stars 355 forks source link

fix custom vllm eval args #2325

Closed Yunnglin closed 4 days ago

Yunnglin commented 4 days ago

PR type

[x] Bug Fix
[ ] New Feature
[ ] Document Updates
[ ] More Models or Datasets Support

PR information

Fix eval dataset with vllm inference backend args. Add following in ms-swift/swift/llm/utils/vllm_utils.py

if self.repetition_penalty is None:
    self.repetition_penalty = 1.0
if self.top_p is None:
    self.top_p = 1.0
if self.top_k is None:
    self.top_k = -1

Experiment results

Paste your experiment result here(if needed).

Jintao-Huang commented 4 days ago

vllm_utils.py可以还原一下不, 怎么改了这么多哇

Yunnglin commented 4 days ago

vllm_utils.py可以还原一下不, 怎么改了这么多哇

自动lint工具修改的，改了一些格式，vllm_utils.py还原了

Jintao-Huang commented 4 days ago

thank you for your PR!