how to generate a llama.cpp server with fastchat api

vicuna-tools / vicuna-installation-guide

The "vicuna-installation-guide" provides step-by-step instructions for installing and configuring Vicuna 13 and 7B

285 stars 34 forks source link

how to generate a llama.cpp server with fastchat api #16

Open xx-zhang opened 1 year ago

xx-zhang commented 1 year ago

I have set the server , but only few words output like blocked , and is a single progress which can't reponse fastly. it is only run and load model when the request is getting.

fredi-python commented 1 year ago

How did you setup the server?