search

meta-math / MetaMath

MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
https://meta-math.github.io
Apache License 2.0
387 stars 35 forks source link

May I know how you load QLoRA+LLaMA 70B with vllm? #15

Closed SuperBruceJia closed 11 months ago

yulonghui commented 11 months ago

Hi, We first merge the qlora weights and 70B base weights by peft repo. the final weights are saved in a local dir. After that, you can load with vllm smoothly