SJTU-IPADS / PowerInfer

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

MIT License

7.97k stars 415 forks source link

about the use of OPT model #228

Open bobzhang208 opened 1 month ago

bobzhang208 commented 1 month ago

Prerequisites

Before submitting your question, please ensure the following:

[ x] I am running the latest version of PowerInfer. Development is rapid, and as of now, there are no tagged versions.
[ x] I have carefully read and followed the instructions in the README.md.
[x ] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).

Question Details

Please provide a clear and concise description of your question. If applicable, include steps to reproduce the issue or behaviors you've observed.

Additional Context

how to use OPT model with powerinfer