search

NexaAI / nexa-sdk

Nexa SDK is a comprehensive toolkit for supporting ONNX and GGML models. It supports text generation, image generation, vision-language models (VLM), auto-speech-recognition (ASR), and text-to-speech (TTS) capabilities.
https://docs.nexa.ai/
Apache License 2.0
3.03k stars 441 forks source link

local model #76

Closed ayttop closed 1 month ago

ayttop commented 1 month ago

how to run nexa with locall modele?

zhiyuan8 commented 1 month ago
  1. find local file nexa list
image
  1. find the model you downloaded locally and then run it with full name nexa run gemma-2b:q4_0
image
ayttop commented 1 month ago

thank you

ayttop commented 1 month ago

How do I turn on the server? I tried many commands and they do not work.

ayttop commented 1 month ago

(2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions {"detail":[{"type":"missing","loc":["body"],"msg":"Field required","input":null}]} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"prompt\":\"Tell me a story\"}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl http://127.0.0.1:8000/v1/chat/completions { "messages": [ { "role": "user", "content": "Tell me a story" } ], "max_tokens": 128, "temperature": 0.1, "stream": ture, "stop_words": [] } {"detail":"Method Not Allowed"}curl: (3) unmatched brace in URL position 1: { ^

(2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/chat/completions \ {"detail":[{"type":"missing","loc":["body"],"msg":"Field required","input":null}]}curl: (3) URL rejected: Bad hostname

(2) C:\Users\ArabTech\Desktop\2>-H "Content-Type: application/json" \ '-H' is not recognized as an internal or external command, operable program or batch file.

(2) C:\Users\ArabTech\Desktop\2>-d '{"messages": [{"role": "user", "content": "Tell me a story"}], "max_tokens": 128, "temperature": 0.1, "stream": true, "stop_words": []}' '-d' is not recognized as an internal or external command, operable program or batch file.

(2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{"messages": [{"role": "user", "content": "Tell me a story"}], "max_tokens": 128, "temperature": 0.1, "stream": true, "stop_words": []}' {"detail":[{"type":"json_invalid","loc":["body",0],"msg":"JSON decode error","input":{},"ctx":{"error":"Expecting value"}}]}curl: (3) URL rejected: Bad hostname curl: (3) URL rejected: Bad hostname curl: (3) bad range specification in URL position 2: [{role: ^

(2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/chat/completions ^ More? -H "Content-Type: application/json" ^ More? -d "{\"messages\": [{\"role\": \"user\", \"content\": \"Tell me a story\"}], \"max_tokens\": 128, \"temperature\": 0.1, \"stream\": true, \"stop_words\": []}" curl: (18) transfer closed with outstanding read data remaining

(2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/chat/completions ^ More? -H "Content-Type: application/json" ^ More? -d "{\"messages\": [{\"role\": \"user\", \"content\": \"Tell me a story\"}], \"max_tokens\": 128, \"temperature\": 0.1, \"stream\": true, \"stop_words\": []}" curl: (18) transfer closed with outstanding read data remaining

(2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/chat/completions ^ More? -H "Content-Type: application/json" ^ More? -d "{\"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/chat/completions ^ More? -H "Content-Type: application/json" ^ More? -d "{\"text\": \"Tell me a story\"}" {"detail":"'dict' object has no attribute 'content'"} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/chat/completions ^ More? -H "Content-Type: application/json" ^ More? -d "{\"text\": \"Hello\"}" {"detail":"'dict' object has no attribute 'content'"} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/chat/completions ^ More? -H "Content-Type: application/json" ^ More? -d "{\"text\": \"Tell me a story\"}" {"detail":"'dict' object has no attribute 'content'"} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/chat/completions ^ More? -H "Content-Type: application/json" ^ More? -d "{\"messages\": [{\"role\": \"user\", \"content\": \"Tell me a story\"}], \"max_tokens\": 128, \"temperature\": 0.1, \"stream\": true, \"stop_words\": []}" curl: (18) transfer closed with outstanding read data remaining

(2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/chat/completions ^ More? -H "Content-Type: application/json" ^ More? -d "{\"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl -v -X POST http://127.0.0.1:8000/v1/chat/completions ^ More? -H "Content-Type: application/json" ^ More? -d "{\"messages\": [{\"role\": \"user\", \"content\": \"Tell me a story\"}], \"max_tokens\": 128, \"temperature\": 0.1, \"stream\": true, \"stop_words\": []}" Note: Unnecessary use of -X or --request, POST is already inferred.

(2) C:\Users\ArabTech\Desktop\2>curl -v --no-buffer -X POST http://127.0.0.1:8000/v1/chat/completions ^ More? -H "Content-Type: application/json" ^ More? -d "{\"messages\": [{\"role\": \"user\", \"content\": \"Tell me a story\"}], \"max_tokens\": 128, \"temperature\": 0.1, \"stream\": true, \"stop_words\": []}" Note: Unnecessary use of -X or --request, POST is already inferred.

(2) C:\Users\ArabTech\Desktop\2>curl -v -X POST http://127.0.0.1:8000/v1/chat/completions

(2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -d "{\"messages\": [{\"role\": \"user\", \"content\": \"Tell me a story\"}], \"max_tokens\": 128, \"temperature\": 0.1, \"stream\": true, \"stop_words\": []}" curl: (18) transfer closed with outstanding read data remaining

(2) C:\Users\ArabTech\Desktop\2>http POST http://127.0.0.1:8000/v1/chat/completions Content-Type:application/json messages:='[{"role": "user", "content": "Tell me a story"}]' max_tokens=128 temperature=0.1 stream=true stop_words:=[] 'http' is not recognized as an internal or external command, operable program or batch file.

(2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -d "{\"messages\": [{\"role\": \"user\", \"content\": \"Tell me a story\"}], \"max_tokens\": 128, \"temperature\": 0.1, \"stream\": true, \"stop_words\": []}" -o response.json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 135 0 0 100 135 0 1110 --:--:-- --:--:-- --:--:-- 1115 curl: (18) transfer closed with outstanding read data remaining

(2) C:\Users\ArabTech\Desktop\2>type response.json The system cannot find the file specified.

(2) C:\Users\ArabTech\Desktop\2>(2) C:\Users\ArabTech\Desktop\2>type response.json C:\Users\ArabTech\Desktop\2 was unexpected at this time.

(2) C:\Users\ArabTech\Desktop\2>The system cannot find the file specified. 'The' is not recognized as an internal or external command, operable program or batch file.

(2) C:\Users\ArabTech\Desktop\2>curl --http1.1 -X POST http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -d "{\"messages\": [{\"role\": \"user\", \"content\": \"Tell me a story\"}], \"max_tokens\": 128, \"temperature\": 0.1, \"stream\": true, \"stop_words\": []}" -o response.json % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 135 0 0 100 135 0 1149 --:--:-- --:--:-- --:--:-- 1153 curl: (18) transfer closed with outstanding read data remaining

(2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -d "{\"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>Invoke-RestMethod -Uri http://127.0.0.1:8000/v1/chat/completions -Method Post -Headers @{ "Content-Type" = "application/json" } -Body '{"messages": [{"role": "user", "content": "Tell me a story"}], "max_tokens": 128, "temperature": 0.1, "stop_words": []}' 'Invoke-RestMethod' is not recognized as an internal or external command, operable program or batch file.

(2) C:\Users\ArabTech\Desktop\2>

ayttop commented 1 month ago

(2) C:\Users\ArabTech\Desktop\2>nexa server --host 127.0.0.1 --port 8000 Phi-2:q4_0 INFO: Started server process [4388] INFO: Waiting for application startup. 2024-09-09 15:48:52,387 - INFO - Model Path: Phi-2:q4_0 Model Phi-2:q4_0 already exists at C:\Users\ArabTech.cache\nexa\hub\official\Phi-2\q4_0.gguf 2024-09-09 15:48:53,043 - INFO - model loaded as <nexa.gguf.llama.llama.Llama object at 0x0000021258A5E050> INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit) INFO: 127.0.0.1:55320 - "POST /v1/completions HTTP/1.1" 422 Unprocessable Entity INFO: 127.0.0.1:55329 - "POST /v1/completions HTTP/1.1" 422 Unprocessable Entity 2024-09-09 15:52:57,436 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55342 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error 2024-09-09 15:53:13,902 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55346 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error 2024-09-09 15:53:21,474 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55353 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error 2024-09-09 15:53:40,359 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55370 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error 2024-09-09 15:54:01,041 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55373 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error

(base) C:\Windows\System32>conda activate 2 && cd C:\Users\ArabTech\Desktop\2

(2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions {"detail":[{"type":"missing","loc":["body"],"msg":"Field required","input":null}]} (2) C:\Users\ArabTech\Desktop\2>curl --location "http://127.0.0.1:8000/v1/completions" --data "{\"model\":\"Phi-2:q4_0\",\"prompt\":\"What is the meaning of life?\",\"system\":\"Speak like a pirate\",\"stream\":false,\"temperature\":0.7,\"top_n\":40}" {"detail":[{"type":"model_attributes_type","loc":["body"],"msg":"Input should be a valid dictionary or object to extract fields from","input":"{\"model\":\"Phi-2:q4_0\",\"prompt\":\"What is the meaning of life?\",\"system\":\"Speak like a pirate\",\"stream\":false,\"temperature\":0.7,\"top_n\":40}"}]} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"prompt\":\"What is the meaning of life?\"}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"text\":\"What is the meaning of life?\"}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"text\":\"What is the meaning of life?\", \"model\":\"Phi-2:q4_0\"}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"text\":\"What is the meaning of life?\"}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"text\":\"What is the meaning of life?\",\"model\":\"some-model\"}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>

ayttop commented 1 month ago

(base) C:\Windows\System32>conda activate 2 && cd C:\Users\ArabTech\Desktop\2

(2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions {"detail":[{"type":"missing","loc":["body"],"msg":"Field required","input":null}]} (2) C:\Users\ArabTech\Desktop\2>curl --location "http://127.0.0.1:8000/v1/completions" --data "{\"model\":\"Phi-2:q4_0\",\"prompt\":\"What is the meaning of life?\",\"system\":\"Speak like a pirate\",\"stream\":false,\"temperature\":0.7,\"top_n\":40}" {"detail":[{"type":"model_attributes_type","loc":["body"],"msg":"Input should be a valid dictionary or object to extract fields from","input":"{\"model\":\"Phi-2:q4_0\",\"prompt\":\"What is the meaning of life?\",\"system\":\"Speak like a pirate\",\"stream\":false,\"temperature\":0.7,\"top_n\":40}"}]} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"prompt\":\"What is the meaning of life?\"}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"text\":\"What is the meaning of life?\"}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"text\":\"What is the meaning of life?\", \"model\":\"Phi-2:q4_0\"}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"text\":\"What is the meaning of life?\"}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"text\":\"What is the meaning of life?\",\"model\":\"some-model\"}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl http://127.0.0.1:8000

Welcome to Nexa AI

Hostname: DESKTOP-86OK6E9

(2) C:\Users\ArabTech\Desktop\2>curl -X POST -d "param1=value1¶m2=value2" https://api.example.com/submit curl: (6) Could not resolve host: api.example.com (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000 -d "param1=value1¶m2=value2" -H "Content-Type: application/x-www-form-urlencoded" -o response.txt % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 58 100 31 100 27 3952 3442 --:--:-- --:--:-- --:--:-- 8285 (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions \ {"detail":[{"type":"missing","loc":["body"],"msg":"Field required","input":null}]}curl: (3) URL rejected: Bad hostname (2) C:\Users\ArabTech\Desktop\2>-H "Content-Type: application/json" \ '-H' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2>-d '{ '-d' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2> "prompt": "Tell me a story", '"prompt":' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2> "temperature": 1, '"temperature":' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2> "max_new_tokens": 128, '"max_new_tokens":' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2> "top_k": 50, '"top_k":' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2> "top_p": 1, '"top_p":' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2> "stop_words": ["string"] '"stop_words":' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2>}' '}'' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions \ {"detail":[{"type":"missing","loc":["body"],"msg":"Field required","input":null}]}curl: (3) URL rejected: Bad hostname (2) C:\Users\ArabTech\Desktop\2>-H "Content-Type: application/json" \ '-H' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2>-d '{ '-d' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2> "prompt": "Tell me a story", '"prompt":' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2> "temperature": 1, '"temperature":' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2> "max_new_tokens": 128, '"max_new_tokens":' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2> "top_k": 50, '"top_k":' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2> "top_p": 1, '"top_p":' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2> "stop_words": ["string"] '"stop_words":' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2>}' '}'' is not recognized as an internal or external command, operable program or batch file. (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d '{"prompt": "Tell me a story", "temperature": 1, "max_new_tokens": 128, "top_k": 50, "top_p": 1, "stop_words": ["string"]}'[^1^][1] {"detail":[{"type":"json_invalid","loc":["body",0],"msg":"JSON decode error","input":{},"ctx":{"error":"Expecting value"}}]}curl: (3) URL rejected: Malformed input to a URL function curl: (3) URL rejected: Port number was not a decimal number between 0 and 65535 curl: (3) URL rejected: Bad hostname curl: (3) URL rejected: Port number was not a decimal number between 0 and 65535 curl: (3) URL rejected: Bad hostname curl: (3) URL rejected: Port number was not a decimal number between 0 and 65535 curl: (3) URL rejected: Bad hostname curl: (3) URL rejected: Port number was not a decimal number between 0 and 65535 curl: (3) URL rejected: Bad hostname curl: (3) URL rejected: Port number was not a decimal number between 0 and 65535 curl: (3) bad range in URL position 2: [string]}'[1][1] ^ (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d '{"prompt": "Tell me a story", "temperature": 1, "max_new_tokens": 128, "top_k": 50, "top_p": 1, "stop_words": ["string"]}' {"detail":[{"type":"json_invalid","loc":["body",0],"msg":"JSON decode error","input":{},"ctx":{"error":"Expecting value"}}]}curl: (3) URL rejected: Malformed input to a URL function curl: (3) URL rejected: Port number was not a decimal number between 0 and 65535 curl: (3) URL rejected: Bad hostname curl: (3) URL rejected: Port number was not a decimal number between 0 and 65535 curl: (3) URL rejected: Bad hostname curl: (3) URL rejected: Port number was not a decimal number between 0 and 65535 curl: (3) URL rejected: Bad hostname curl: (3) URL rejected: Port number was not a decimal number between 0 and 65535 curl: (3) URL rejected: Bad hostname curl: (3) URL rejected: Port number was not a decimal number between 0 and 65535 curl: (3) bad range in URL position 2: [string]}' ^ (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"prompt\": \"Tell me a story\", \"temperature\": 1, \"max_new_tokens\": 128, \"top_k\": 50, \"top_p\": 1, \"stop_words\": [\"string\"]}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"prompt\": \"Tell me a story\"}" {"detail":"'text'"} (2) C:\Users\ArabTech\Desktop\2>curl -v -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"prompt\": \"Tell me a story\", \"temperature\": 1, \"max_new_tokens\": 128, \"top_k\": 50, \"top_p\": 1, \"stop_words\": [\"string\"]}" Note: Unnecessary use of -X or --request, POST is already inferred. * Trying 127.0.0.1:8000... * Connected to 127.0.0.1 (127.0.0.1) port 8000 > POST /v1/completions HTTP/1.1 > Host: 127.0.0.1:8000 > User-Agent: curl/8.8.0 > Accept: */* > Content-Type: application/json > Content-Length: 121 > * upload completely sent off: 121 bytes < HTTP/1.1 500 Internal Server Error < date: Mon, 09 Sep 2024 23:06:52 GMT < server: uvicorn < content-length: 19 < content-type: application/json < {"detail":"'text'"}* Connection #0 to host 127.0.0.1 left intact (2) C:\Users\ArabTech\Desktop\2>curl -v -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"text\": \"Tell me a story\", \"temperature\": 1, \"max_new_tokens\": 128, \"top_k\": 50, \"top_p\": 1, \"stop_words\": [\"string\"]}" Note: Unnecessary use of -X or --request, POST is already inferred. * Trying 127.0.0.1:8000... * Connected to 127.0.0.1 (127.0.0.1) port 8000 > POST /v1/completions HTTP/1.1 > Host: 127.0.0.1:8000 > User-Agent: curl/8.8.0 > Accept: */* > Content-Type: application/json > Content-Length: 119 > * upload completely sent off: 119 bytes < HTTP/1.1 500 Internal Server Error < date: Mon, 09 Sep 2024 23:07:14 GMT < server: uvicorn < content-length: 19 < content-type: application/json < {"detail":"'text'"}* Connection #0 to host 127.0.0.1 left intact (2) C:\Users\ArabTech\Desktop\2>curl -v -X POST http://127.0.0.1:8000/v1/completions -H "Content-Type: application/json" -d "{\"text\": \"Tell me a story\"}" Note: Unnecessary use of -X or --request, POST is already inferred. * Trying 127.0.0.1:8000... * Connected to 127.0.0.1 (127.0.0.1) port 8000 > POST /v1/completions HTTP/1.1 > Host: 127.0.0.1:8000 > User-Agent: curl/8.8.0 > Accept: */* > Content-Type: application/json > Content-Length: 27 > * upload completely sent off: 27 bytes < HTTP/1.1 500 Internal Server Error < date: Mon, 09 Sep 2024 23:07:32 GMT < server: uvicorn < content-length: 19 < content-type: application/json < {"detail":"'text'"}* Connection #0 to host 127.0.0.1 left intact (2) C:\Users\ArabTech\Desktop\2>
ayttop commented 1 month ago

(2) C:\Users\ArabTech\Desktop\2>nexa server --host 127.0.0.1 --port 8000 Phi-2:q4_0 INFO: Started server process [4388] INFO: Waiting for application startup. 2024-09-09 15:48:52,387 - INFO - Model Path: Phi-2:q4_0 Model Phi-2:q4_0 already exists at C:\Users\ArabTech.cache\nexa\hub\official\Phi-2\q4_0.gguf 2024-09-09 15:48:53,043 - INFO - model loaded as <nexa.gguf.llama.llama.Llama object at 0x0000021258A5E050> INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit) INFO: 127.0.0.1:55320 - "POST /v1/completions HTTP/1.1" 422 Unprocessable Entity INFO: 127.0.0.1:55329 - "POST /v1/completions HTTP/1.1" 422 Unprocessable Entity 2024-09-09 15:52:57,436 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55342 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error 2024-09-09 15:53:13,902 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55346 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error 2024-09-09 15:53:21,474 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55353 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error 2024-09-09 15:53:40,359 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55370 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error 2024-09-09 15:54:01,041 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55373 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error INFO: 127.0.0.1:55469 - "GET / HTTP/1.1" 200 OK INFO: 127.0.0.1:55506 - "POST / HTTP/1.1" 405 Method Not Allowed INFO: 127.0.0.1:55544 - "POST /v1/completions HTTP/1.1" 422 Unprocessable Entity INFO: 127.0.0.1:55548 - "POST /v1/completions HTTP/1.1" 422 Unprocessable Entity INFO: 127.0.0.1:55556 - "POST /v1/completions HTTP/1.1" 422 Unprocessable Entity INFO: 127.0.0.1:55571 - "POST /v1/completions HTTP/1.1" 422 Unprocessable Entity 2024-09-09 16:06:39,494 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55581 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error 2024-09-09 16:06:46,817 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55585 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error 2024-09-09 16:06:53,425 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55589 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error 2024-09-09 16:07:14,735 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55592 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error 2024-09-09 16:07:33,011 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55596 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error 2024-09-09 16:09:13,749 - ERROR - Error in text generation: 'text' INFO: 127.0.0.1:55636 - "POST /v1/completions HTTP/1.1" 500 Internal Server Error

JoyboyBrian commented 1 month ago

Hi, @ayttop! It appears you've successfully started a server, but there seem to be issues with the way requests are being made. I recommend accessing http://127.0.0.1:8000/docs to run some tests and examine the structure of the payload. This should help you identify the correct format for your requests and resolve the errors you're encountering.

JoyboyBrian commented 1 month ago

https://github.com/user-attachments/assets/9b24c8f3-5049-4549-888b-46adf3345477

ayttop commented 1 month ago

Thank you It work

From: Brian @.> Sent: Monday, September 9, 2024 4:29 PM To: NexaAI/nexa-sdk @.> Cc: ayttop @.>; Mention @.> Subject: Re: [NexaAI/nexa-sdk] local model (Issue #76)

https://github.com/user-attachments/assets/9b24c8f3-5049-4549-888b-46adf3345477

— Reply to this email directly, view it on GitHubhttps://github.com/NexaAI/nexa-sdk/issues/76#issuecomment-2339325829, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BKTFRESEXKLH3JDFIJUKUDTZVYVN3AVCNFSM6AAAAABN5LDINCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZZGMZDKOBSHE. You are receiving this because you were mentioned.Message ID: @.***>

JoyboyBrian commented 1 month ago

Great!