Compatibility with other selfhosted ChatGPT alternatives

jina-ai / thinkgpt

Agent techniques to augment your LLM and push it beyong its limits

Apache License 2.0

1.53k stars 134 forks source link

Compatibility with other selfhosted ChatGPT alternatives #15

Closed UmutAlihan closed 1 month ago

UmutAlihan commented 1 year ago

Hi,

Your contribution to OS community is greatly appreciated. Thank you!

I am curious that whether this repo is only for OpenAI API?

Is it possible to use with Falcon-40B-Instruct model? If so how? Could you provide some help?

Cheers,

crrrr30 commented 1 year ago

I've been working on the same thing, and the best solution I found so far is to use vLLM to host an OpenAI-like server and modify this repository to use VLLMOpenAI instead, calling that server.