Support converting TurboSparse mistral model into PowerInfer GGUF

SJTU-IPADS / PowerInfer

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

MIT License

7.97k stars 415 forks source link

Open hodlen opened 4 months ago

hodlen commented 4 months ago

Support converting TurboSparse mistral model which embeds MLP in Pytorch tensors.