Support for LLama and Hugging Face models?

minimaxir / simpleaichat

Python package for easily interfacing with chat apps, with robust features and minimal code complexity.

MIT License

3.43k stars 224 forks source link

Support for LLama and Hugging Face models? #81

Closed sysadminpower2019 closed 9 months ago

sysadminpower2019 commented 10 months ago

Is this currently supported?

minimaxir commented 9 months ago

Not currently; it's in the pipeline for investigation, but there's no way to make it "simple" because there's a very large number of software and hardware constraints so it's not a trivial task.

sysadminpower2019 commented 9 months ago

is there a way to use the simpleaichat then without a chatgpt / openai api key? I would rather not feed the OpenAI monster even more data, if we can instead find a way to self host things

adrianog commented 9 months ago

can you not use a adapter to adapt hf models to OAI interface?