search

AyoOdumark / llama-2-server

0 stars 0 forks source link

Testing other LLMs #2

Open krrishdholakia opened 1 year ago

krrishdholakia commented 1 year ago

Hi @Akintunde102 @AyoOdumark ,

if you're trying to test other LLMs (codellama, wizardcoder, etc.) with GPT-Engineer, we just open-sourced a 1-click proxy to translate openai calls to huggingface, anthropic, togetherai, etc. api calls.

code

$ pip install litellm

$ litellm --model huggingface/bigcode/starcoder

#INFO:     Uvicorn running on http://0.0.0.0:8000

Docs: https://docs.litellm.ai/docs/proxy_server

I'd love to know if this solves any problem for you

AyoOdumark commented 1 year ago

We will check that out

krrishdholakia commented 1 year ago

Thanks - was testing new llms a problem for you before?