search

gorilla-llm / gorilla-cli

LLMs for your CLI
https://gorilla.cs.berkeley.edu/
Apache License 2.0
1.28k stars 75 forks source link

Opensource backend code #33

Closed nmiletic closed 1 year ago

nmiletic commented 1 year ago

Thank you for this very interesting project. Is backend code also going open source?

ShishirPatil commented 1 year ago

Thanks you @nmiletic. Not yet, but I will clean it up and upload later. It is basically an uvicorn server that receives the request, and dispatches inference requests to another uvicorn server that is marshalling the LLM models on the GPU. Hope this helps :)