search

mMrBun / AIPC

Apache License 2.0
57 stars 19 forks source link

想问下dtype可以默认改成float16吗 #26

Closed Teryua closed 3 days ago

Teryua commented 4 months ago

麻烦再问下，vllm推理的dtype默认精度怎么改成float16

mMrBun commented 3 days ago

vllm启动参数里有dtype的设置