search

kubeagi / arcadia

A diverse, simple, and secure one-stop LLMOps platform
http://www.kubeagi.com/
Apache License 2.0
63 stars 20 forks source link

feat: add http debug client for llm call #1004

Closed Abirdcfly closed 2 months ago

Abirdcfly commented 2 months ago

What type of PR is this?

/kind feature

What this PR does / why we need it

Which issue(s) this PR fixes

Fixes #

Special notes for your reviewer

log example:

I0410 07:06:48.141884       1 app_runtime.go:197] "try to run node:prompt-node" requestID="d857dcdb2ef313009c8a65aa3481b7ff"
I0410 07:06:48.141897       1 app_runtime.go:197] "try to run node:chain-node" requestID="d857dcdb2ef313009c8a65aa3481b7ff"
I0410 07:06:48.142083       1 log.go:53] "Entering LLM with messages: text: Human: \nnihRole: human" requestID="d857dcdb2ef313009c8a65aa3481b7ff"
I0410 07:06:48.142302       1 debug_transport.go:26] "POST /v1/chat/completions HTTP/1.1\r\nHost: arcadia-fastchat.kubeagi-system.svc.cluster.local:8000\r\nUser-Agent: Go-http-client/1.1\r\nContent-Length: 153\r\nAuthorization: Bearer fake\r\nContent-Type: application/json\r\nAccept-Encoding: gzip\r\n\r\n{\"model\":\"chatglm_lite\",\"messages\":[{\"role\":\"user\",\"content\":[{\"text\":\"Human: \\nnih\",\"type\":\"text\"}]}],\"temperature\":0.7,\"max_tokens\":2048,\"stream\":true}" requestID="d857dcdb2ef313009c8a65aa3481b7ff"
I0410 07:06:48.147648       1 debug_transport.go:35] "HTTP/1.1 400 Bad Request\r\nContent-Length: 160\r\nContent-Type: application/json\r\nDate: Wed, 10 Apr 2024 07:06:47 GMT\r\nServer: uvicorn\r\n\r\n{\"object\":\"error\",\"message\":\"Only cea10419-c146-4ba9-bf80-1db2eabf35b8&&7d862d32-a19d-42b1-b4ef-f2ef64aceeb0 allowed now, your model chatglm_lite\",\"code\":40301}" requestID="d857dcdb2ef313009c8a65aa3481b7ff"
I0410 07:06:48.147725       1 llmchain.go:154] "use llmchain, blocking out:" requestID="d857dcdb2ef313009c8a65aa3481b7ff"
E0410 07:06:48.147828       1 chat.go:116] "error resp, stop the stream" err="run node chain-node: llmchain run error: API returned unexpected status code: 400: " requestID="d857dcdb2ef313009c8a65aa3481b7ff"