search

databricks / dbrx

Code examples and resources for DBRX, a large language model developed by Databricks
https://www.databricks.com/
Other
2.47k stars 231 forks source link

Quantized distilled version #15

Closed florian-hoenicke closed 3 months ago

florian-hoenicke commented 3 months ago

It would be great to have a version that can be better worked with locally - ideally executable on CPU

hanlint commented 3 months ago

Hi @florian-hoenicke , unfortunately this model is quite large and will not be performant on a CPU. We are considering building a smaller version, and will take your input into consideration.

If you have Apple M2 laptop, you may want to try MLX, where DBRX example was recently added: https://github.com/ml-explore/mlx-examples/pull/628