Closed avinashkr29 closed 1 year ago
I was able to solve the issue by changing the final command in colab to the following:
!pip install flask-cloudflared
!python server.py --api --public-api --share --settings /content/settings.yaml --wbits 4 --groupsize 128 --loader AutoGPTQ --model /content/text-generation-webui/models/vicuna-13b-GPTQ-4bit-128g
After checking the "api" option under the Session tab, I clicked the "Apply flags/extension and Restart" button as shown below:
This generated the following logs in the colab console:
I tried the following code to get the response after that. However, I am getting 404 error. Could you please tell me how do I start the API correctly and get the responses?