Question about deployment of fine tuned model

artidoro / qlora

QLoRA: Efficient Finetuning of Quantized LLMs

https://arxiv.org/abs/2305.14314

MIT License

9.74k stars 800 forks source link

Question about deployment of fine tuned model #289

Open Brandon371 opened 4 months ago

Brandon371 commented 4 months ago

I have used Qlora to fine tune the LLM model. I have seen some models on output folder. How can I run the output model and test them? I have tried to use some open source API. It seems requiring config.json to deploy this model on my server. How can I deploy the fine tuned model locally?