search

katanemo / arch

Arch is an intelligent prompt gateway. Engineered with (fast) LLMs for the secure handling, robust observability, and seamless integration of prompts with APIs - all outside business logic. Built by the core contributors of Envoy proxy, on Envoy.
https://archgw.com
Apache License 2.0
438 stars 23 forks source link

Network Agent demo doesn't respond #232

Open dougalb opened 1 day ago

dougalb commented 1 day ago

Setup arch on MacOS 15.1 with Docker. Following steps for Network Agent demo and after entering tell me what you can help me with in the UI, nothing happens.

Logs from archgw logs --follow below:

archgw-1  | [2024-10-30 15:44:57.207][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: ||| create_http_context called with context_id: 3 |||
archgw-1  | [2024-10-30 15:44:57.207][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: llm provider hint: None
archgw-1  | [2024-10-30 15:44:57.208][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: no llm provider found for hint, using default llm provider
archgw-1  | [2024-10-30 15:44:57.208][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: selected llm: OpenAI
archgw-1  | [2024-10-30 15:44:57.208][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: S[3] req_headers=[(":authority", "host.docker.internal:10000"), (":path", "/v1/chat/completions"), (":method", "POST"), (":scheme", "https"), ("accept-encoding", "gzip, deflate"), ("accept", "application/json"), ("content-type", "application/json"), ("user-agent", "OpenAI/Python 1.51.0"), ("x-stainless-lang", "python"), ("x-stainless-package-version", "1.51.0"), ("x-stainless-os", "Linux"), ("x-stainless-arch", "arm64"), ("x-stainless-runtime", "CPython"), ("x-stainless-runtime-version", "3.10.15"), ("x-stainless-async", "false"), ("x-stainless-retry-count", "0"), ("x-forwarded-proto", "https"), ("x-request-id", "e85ca1c1-53ee-4bdd-b074-7fa7b01988d8"), ("x-arch-upstream", "arch_llm_listener"), ("authorization", "Bearer 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")]
archgw-1  | [2024-10-30 15:44:57.209][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: Missing input guard. Making inline call to retrieve
archgw-1  | [2024-10-30 15:44:57.209][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: dispatching http call with args=CallArgs { upstream: "arch_internal", path: "/embeddings", headers: [("x-arch-upstream", "model_server"), (":method", "POST"), (":path", "/embeddings"), (":authority", "model_server"), ("content-type", "application/json"), ("x-envoy-max-retries", "3"), ("x-envoy-upstream-rq-timeout-ms", "60000"), ("x-request-id", "e85ca1c1-53ee-4bdd-b074-7fa7b01988d8")], trailers: [], timeout: 5s } context=StreamCallContext { response_handler_type: GetEmbeddings, user_message: Some("tell me what you can help me with?"), prompt_target_name: None, request_body: ChatCompletionsRequest { model: "gpt-3.5-turbo", messages: [Message { role: "user", content: Some("tell me what you can help me with?"), model: None, tool_calls: None }], tools: None, stream: true, stream_options: Some(StreamOptions { include_usage: true }), metadata: None }, tool_calls: None, similarity_scores: None, upstream_cluster: None, upstream_cluster_path: None }
archgw-1  | [2024-10-30 15:44:57.210][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: ||| create_http_context called with context_id: 3 |||
archgw-1  | [2024-10-30 15:44:57.210][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: llm provider hint: None
archgw-1  | [2024-10-30 15:44:57.210][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: no llm provider found for hint, using default llm provider
archgw-1  | [2024-10-30 15:44:57.210][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: selected llm: OpenAI
archgw-1  | [2024-10-30 15:44:57.210][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: S[3] req_headers=[(":authority", "arch_llm_listener"), (":path", "/v1/chat/completions"), (":method", "POST"), (":scheme", "https"), ("accept-encoding", "gzip, deflate"), ("accept", "application/json"), ("content-type", "application/json"), ("user-agent", "OpenAI/Python 1.51.0"), ("x-stainless-lang", "python"), ("x-stainless-package-version", "1.51.0"), ("x-stainless-os", "Linux"), ("x-stainless-arch", "arm64"), ("x-stainless-runtime", "CPython"), ("x-stainless-runtime-version", "3.10.15"), ("x-stainless-async", "false"), ("x-stainless-retry-count", "0"), ("x-forwarded-proto", "https"), ("x-request-id", "e85ca1c1-53ee-4bdd-b074-7fa7b01988d8"), ("x-arch-upstream", "arch_llm_listener"), ("x-arch-llm-provider", "OpenAI"), ("authorization", "Bearer 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")]
2024-10-30 08:44:57,219 - INFO - Embedding req: input='tell me what you can help me with?' model='katanemo/bge-large-en-v1.5'
2024-10-30 08:44:57,287 - INFO - Embedding Call Complete Time: 0.06710670800021035
archgw-1  | [2024-10-30 15:44:57.290][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: embedding model: katanemo/bge-large-en-v1.5, vector length: 1024
archgw-1  | [2024-10-30 15:44:57.290][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: similarity scores based on description embeddings match: [("reboot_devices", 0.457049393984019), ("device_summary", 0.4756624797098869)]
archgw-1  | [2024-10-30 15:44:57.290][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: dispatching http call with args=CallArgs { upstream: "arch_internal", path: "/zeroshot", headers: [("x-arch-upstream", "model_server"), (":method", "POST"), (":path", "/zeroshot"), (":authority", "model_server"), ("content-type", "application/json"), ("x-envoy-max-retries", "3"), ("x-envoy-upstream-rq-timeout-ms", "60000"), ("x-request-id", "e85ca1c1-53ee-4bdd-b074-7fa7b01988d8")], trailers: [], timeout: 5s } context=StreamCallContext { response_handler_type: ZeroShotIntent, user_message: Some("tell me what you can help me with?"), prompt_target_name: None, request_body: ChatCompletionsRequest { model: "gpt-3.5-turbo", messages: [Message { role: "user", content: Some("tell me what you can help me with?"), model: None, tool_calls: None }], tools: None, stream: true, stream_options: Some(StreamOptions { include_usage: true }), metadata: None }, tool_calls: None, similarity_scores: Some([("reboot_devices", 0.457049393984019), ("device_summary", 0.4756624797098869)]), upstream_cluster: None, upstream_cluster_path: None }
2024-10-30 08:44:57,293 - INFO - zero-shot request: input='tell me what you can help me with?' labels=['reboot_devices', 'device_summary'] model='katanemo/bart-large-mnli'
2024-10-30 08:44:57,787 - INFO - zero-shot taking 0.49322237500018673 seconds
archgw-1  | [2024-10-30 15:44:57.787][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: zeroshot intent response: ZeroShotClassificationResponse { predicted_class: "device_summary", predicted_class_score: 0.3438539206981659, scores: {"device_summary": 0.3438539206981659, "reboot_devices": 0.16097179055213928}, model: "katanemo/bart-large-mnli" }
archgw-1  | [2024-10-30 15:44:57.787][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: similarity score: 0.383, intent score: 0.344, description embedding score: 0.476, prompt: tell me what you can help me with?
archgw-1  | [2024-10-30 15:44:57.787][23][info][wasm] [source/extensions/common/wasm/context.cc:1195] wasm log http_config prompt_gateway: no assistant message found, probably first interaction
archgw-1  | [2024-10-30 15:44:57.787][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: intent score is low or arch assistant is handling the conversation
archgw-1  | [2024-10-30 15:44:57.787][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: checking for default prompt target
archgw-1  | [2024-10-30 15:44:57.787][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: default prompt target found
archgw-1  | [2024-10-30 15:44:57.787][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: no prompt target found with similarity score above threshold, using default prompt target
archgw-1  | [2024-10-30 15:44:57.788][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: dispatching http call with args=CallArgs { upstream: "arch_internal", path: "/agent/network_summary", headers: [(":method", "POST"), ("x-arch-upstream", "app_server"), (":path", "/agent/network_summary"), (":authority", "app_server"), ("content-type", "application/json"), ("x-envoy-max-retries", "3"), ("x-envoy-upstream-rq-timeout-ms", "120000"), ("x-request-id", "e85ca1c1-53ee-4bdd-b074-7fa7b01988d8")], trailers: [], timeout: 5s } context=StreamCallContext { response_handler_type: DefaultTarget, user_message: Some("tell me what you can help me with?"), prompt_target_name: Some("network_qa"), request_body: ChatCompletionsRequest { model: "gpt-3.5-turbo", messages: [Message { role: "user", content: Some("tell me what you can help me with?"), model: None, tool_calls: None }], tools: None, stream: true, stream_options: Some(StreamOptions { include_usage: true }), metadata: None }, tool_calls: None, similarity_scores: Some([("reboot_devices", 0.457049393984019), ("device_summary", 0.4756624797098869)]), upstream_cluster: None, upstream_cluster_path: None }
archgw-1  | [2024-10-30 15:44:57.789][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: llm gateway mode, skipping over all prompt targets
archgw-1  | [2024-10-30 15:44:57.789][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: arch => "gpt-3.5-turbo", body: {"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"tell me what you can help me with?"}],"stream":true}
archgw-1  | [2024-10-30 15:44:57.795][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: response received for default target: network_qa
archgw-1  | [2024-10-30 15:44:57.795][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: sending response back to developer: {"choices":[{"message":{"role":"assistant","content":"I am a helpful networking agent, and I can help you get status for network devices or reboot them"},"finish_reason":"completed","index":0}],"model":"network_agent","usage":{"completion_tokens":0}}
archgw-1  | [2024-10-30 15:44:57.795][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: recv [S=3] bytes=250 end_stream=true
archgw-1  | [2024-10-30 15:44:57.795][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: streaming response
archgw-1  | [2024-10-30 15:44:57.795][23][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config prompt_gateway: server error occurred: parsing error in streaming data
archgw-1  | [2024-10-30 15:44:58.134][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=531 end_stream=false
archgw-1  | [2024-10-30 15:44:58.168][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=531 end_stream=false
archgw-1  | [2024-10-30 15:44:58.216][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=1288 end_stream=false
archgw-1  | [2024-10-30 15:44:58.233][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=2044 end_stream=false
archgw-1  | [2024-10-30 15:44:58.257][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=2803 end_stream=false
archgw-1  | [2024-10-30 15:44:58.303][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=3562 end_stream=false
archgw-1  | [2024-10-30 15:44:58.312][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=4078 end_stream=false
archgw-1  | [2024-10-30 15:44:58.344][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=4844 end_stream=false
archgw-1  | [2024-10-30 15:44:58.348][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=5603 end_stream=false
archgw-1  | [2024-10-30 15:44:58.387][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=6116 end_stream=false
archgw-1  | [2024-10-30 15:44:58.408][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=6630 end_stream=false
archgw-1  | [2024-10-30 15:44:58.435][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=7390 end_stream=false
archgw-1  | [2024-10-30 15:44:58.464][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=7902 end_stream=false
archgw-1  | [2024-10-30 15:44:58.504][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=8659 end_stream=false
archgw-1  | [2024-10-30 15:44:58.544][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=9421 end_stream=false
archgw-1  | [2024-10-30 15:44:58.641][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=10176 end_stream=false
archgw-1  | [2024-10-30 15:44:58.670][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=10932 end_stream=false
archgw-1  | [2024-10-30 15:44:58.690][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=13704 end_stream=false
archgw-1  | [2024-10-30 15:44:58.705][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=14459 end_stream=false
archgw-1  | [2024-10-30 15:44:58.705][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: recv [S=3] bytes=15467 end_stream=true
archgw-1  | [2024-10-30 15:44:58.705][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: non streaming response
archgw-1  | [2024-10-30 15:44:58.705][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: invalid response: data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"role":"assistant","content":"","refusal":null},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"I"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" can"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" help"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" you"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" with"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" a"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" wide"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" range"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" of"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" tasks"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" and"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" questions"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" including"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" providing"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" information"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" offering"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" advice"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" giving"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" recommendations"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" answering"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" queries"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" assisting"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" with"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" problem"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"-solving"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" engaging"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" in"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" conversation"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" and"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" much"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" more"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" Just"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" let"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" me"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" know"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" what"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" you"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" need"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" help"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" with"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" and"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" I"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" will"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" do"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" my"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" best"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" to"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" assist"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":" you"},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null}]}
archgw-1  |
archgw-1  | data: {"id":"chatcmpl-AO5APbZPEb6BikORBvRYfVLIYwrMC","object":"chat.completion.chunk","created":1730303097,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}]}
archgw-1  |
archgw-1  | data: [DONE]
archgw-1  |
archgw-1  |
archgw-1  | [2024-10-30 15:44:58.705][30][debug][wasm] [source/extensions/common/wasm/context.cc:1192] wasm log http_config llm_gateway: server error occurred: expected value at line 1 column 1
salmanap commented 1 day ago

Thanks for putting this issue up. Looking. Last night some of our pypi distributions broke because of a tensorflow update that we didn't catch in testing. We are patch fixing that now and making sure that doesn't happen again. Can you please share the version of the archgw that you are using. archgw --version. This would help us debug this a bit better.

dougalb commented 1 day ago

It was installed from pypi using pip and doesn't have the version command. Check pip, it is 0.0.3. Manually built the CLI from source and it is now:

(.venv) ➜  tools git:(main) ✗ archgw --version
archgw cli version: 0.0.5
(.venv) ➜  tools git:(main) ✗ archgw
Arch (The Intelligent Prompt Gateway) CLI

     _                _
    / \    _ __  ___ | |__
   / _ \  | '__|/ __|| '_ \
  / ___ \ | |  | (__ | | | |
 /_/   \_\|_|   \___||_| |_|

Usage: archgw [OPTIONS] COMMAND [ARGS]...

Options:
  --version  Show the archgw cli version and exit.
  --help     Show this message and exit.

Commands:
  build                    Build Arch from source.
  down                     Stops Arch.
  generate-prompt-targets  Generats prompt_targets from python methods.
  logs                     Stream logs from access logs services.
  up                       Starts Arch.
(.venv) ➜  tools git:(main) ✗

Will retest using the new cli install. Also as a note, the logs are wiring the API keys out. Might be worth looking at obfuscating those things even in debug logs.

salmanap commented 1 day ago

Absolutely fair on the keys bit. Created a new issue https://github.com/katanemo/arch/issues/234 to fix that. Separately, if you build from source right now, the only thing we know that will cause a failure is that Arch-Function model deployment failed last night. Team reverted the change, but we are seeing intermittent failures. Debugging and will post here, when that is fixed

salmanap commented 22 hours ago

Several fixes went in today - Looks like the network_agent should be functional now. Just tested.

image