SJTU-IPADS / PowerInfer

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

MIT License

7.96k stars 412 forks source link

Will we have instruct fine-tuned model support in the future? #164

Open ZeonfaiHo opened 8 months ago

ZeonfaiHo commented 8 months ago

Prerequisites

Before submitting your question, please ensure the following:

[x] I am running the latest version of PowerInfer. Development is rapid, and as of now, there are no tagged versions.
[x] I have carefully read and followed the instructions in the README.md.
[x] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).

Question Details

Please provide a clear and concise description of your question. If applicable, include steps to reproduce the issue or behaviors you've observed.

Will we have instruct fine-tuned model support in the future?

YixinSong-e commented 7 months ago

Hello, you can try Bamboo-DPO now. The template is Vicuna and ChatML. :)