lmstudio-ai / lms

👾 LM Studio CLI
https://lms.dev
MIT License
1.68k stars 138 forks source link

[Feature Request] Support for bitnet.cpp and BitNet models #89

Open dgallitelli opened 1 month ago

dgallitelli commented 1 month ago

bitnet.cpp is the official inference framework for 1-bit LLMs (e.g., BitNet b1.58). It offers a suite of optimized kernels, that support fast and lossless inference of 1.58-bit models on CPU (with NPU and GPU support coming next).

It would be awesome for LMS to support bitnet.cpp and therefore BitNet models, for on-device evaluation.