search

gotzmann / booster

Booster - open accelerator for LLM models. Better inference and debugging for AI hackers
Other
138 stars 6 forks source link

README instructions #1

Closed venuv closed 10 months ago

venuv commented 1 year ago

Can you add some detail on how I could serve GGML models using this on my mac? End goal is to have a local LangChain process consumer the model (in either streaming or calling mode). thx

gotzmann commented 10 months ago

Hey, please check out new README, there all info you need :)