Closed jlcbj closed 3 weeks ago
jlcbj
commented
1 month ago {"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:a1f293c834254b44", "parent _span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:eb15953f8a3d81dc", "operation_name": "WorkerManager.get_model_metadata", "start_time": "2024-10-15 14:02:09.877", "end_time": null, "metadata": null}
WorkerManager.get_model_metadata没有后续打印了
jlcbj
commented
3 weeks ago semaphore=<asyncio.locks.Semaphore object at 0x7f4e68f19c00 [unlocked, value:1]> semaphore=<asyncio.locks.Semaphore object at 0x7f4e68f19c00 [locked]> 原因是找到了
worker_run_data.semaphore这个信号量，某些原因导致它耗光了，锁死llm的所有交互。
{"span_type": "base", "trace_id": "6b8c96d4d9695647cd030fc50aedda78", "span_id": "6b8c96d4d9695647cd030fc50aedda78:9a954dc22eac2e9f", "parent_span_id": "6b8c96d4d9695647cd030fc50aedda78:f1df8a405e88a879", "operation_name": "dbgpt.awel.operator.call_stream.iterate", "start_time": "2024-10-22 08:37:57.048", "end_time": null, "metadata": null}
{"span_type": "base", "trace_id": "6b8c96d4d9695647cd030fc50aedda78", "span_id": "6b8c96d4d9695647cd030fc50aedda78:7478279d08643e49", "parent_span_id": "6b8c96d4d9695647cd030fc50aedda78:9a954dc22eac2e9f", "operation_name": "WorkerManager.get_model_metadata", "start_time": "2024-10-22 08:37:57.048", "end_time": null, "metadata": null}
{"span_type": "base", "trace_id": "6b8c96d4d9695647cd030fc50aedda78", "span_id": "6b8c96d4d9695647cd030fc50aedda78:7478279d08643e49", "parent_span_id": "6b8c96d4d9695647cd030fc50aedda78:9a954dc22eac2e9f", "operation_name": "WorkerManager.get_model_metadata", "start_time": "2024-10-22 08:37:57.048", "end_time": "2024-10-22 08:37:57.050", "metadata": null}
{"span_type": "base", "trace_id": "6b8c96d4d9695647cd030fc50aedda78", "span_id": "6b8c96d4d9695647cd030fc50aedda78:90ac957d284c07b0", "parent_span_id": "6b8c96d4d9695647cd030fc50aedda78:04508085a61a5cae", "operation_name": "WorkerManager.generate_stream", "start_time": "2024-10-22 08:37:57.052", "end_time": null, "metadata": null}
{"span_type": "base", "trace_id": "6b8c96d4d9695647cd030fc50aedda78", "span_id": "6b8c96d4d9695647cd030fc50aedda78:f2990c6519cf6f39", "parent_span_id": "6b8c96d4d9695647cd030fc50aedda78:90ac957d284c07b0", "operation_name": "DefaultModelWorker.async_generate_stream", "start_time": "2024-10-22 08:37:57.052", "end_time": null, "metadata": null}
{"span_type": "base", "trace_id": "6b8c96d4d9695647cd030fc50aedda78", "span_id": "6b8c96d4d9695647cd030fc50aedda78:86d51c37267a7313", "parent_span_id": "6b8c96d4d9695647cd030fc50aedda78:f2990c6519cf6f39", "operation_name": "DefaultModelWorker_call.generate_stream_func", "start_time": "2024-10-22 08:37:57.053", "end_time": null, "metadata": {"is_async_func": true, "llm_adapter": ", link format xxx, table format should be represented with |xxx|xxx|xxx|, and code format with xxx.\n 2.If the information available in the knowledge base is insufficient to answer the question, state clearly: \"The content provided in the knowledge base is not enough to answer this question,\" and avoid making up answers.\n 3.When responding, it is best to summarize the points in the order of 1, 2, 3, And displayed in markdwon format.\n known information: \n 山真国际\n question:\n 在么,when answering, use the same language as the \"user\".\n", "round_index": 0}, {"role": "human", "content": "在么", "round_index": 0}], "temperature": 0.6, "max_new_tokens": 8192, "echo": false, "span_id": "6b8c96d4d9695647cd030fc50aedda78:90ac957d284c07b0", "context": {"stream": true, "cache_enable": false, "user_name": "1", "sys_code": "1840310408651038721", "span_id": "6b8c96d4d9695647cd030fc50aedda78:d8d4eedfe70df046", "chat_mode": "chat_knowledge", "extra": {}}, "convert_to_compatible_format": false, "string_prompt": "system: Based on the known information below, provide users with professional and concise answers to their questions.\nconstraints:\n 1.Ensure to include original markdown formatting elements such as images, links, tables, or code blocks without alteration in the response if they are present in the provided information.\n For example, image format should be
, link format xxx, table format should be represented with |xxx|xxx|xxx|, and code format with xxx.\n 2.If the information available in the knowledge base is insufficient to answer the question, state clearly: \"The content provided in the knowledge base is not enough to answer this question,\" and avoid making up answers.\n 3.When responding, it is best to summarize the points in the order of 1, 2, 3, And displayed in markdwon format.\n known information: \n 山真国际\n question:\n 在么,when answering, use the same language as the \"user\".\n\nhuman: 在么", "prompt_echo_len_char": -1, "has_format_prompt": false, "prompt": "system: Based on the known information below, provide users with professional and concise answers to their questions.\nconstraints:\n 1.Ensure to include original markdown formatting elements such as images, links, tables, or code blocks without alteration in the response if they are present in the provided information.\n For example, image format should be
, link format xxx, table format should be represented with |xxx|xxx|xxx|, and code format with xxx.\n 2.If the information available in the knowledge base is insufficient to answer the question, state clearly: \"The content provided in the knowledge base is not enough to answer this question,\" and avoid making up answers.\n 3.When responding, it is best to summarize the points in the order of 1, 2, 3, And displayed in markdwon format.\n known information: \n 山真国际\n question:\n 在么,when answering, use the same language as the \"user\".\n\nhuman: 在么"}}
{"span_type": "base", "trace_id": "6b8c96d4d9695647cd030fc50aedda78", "span_id": "6b8c96d4d9695647cd030fc50aedda78:86d51c37267a7313", "parent_span_id": "6b8c96d4d9695647cd030fc50aedda78:f2990c6519cf6f39", "operation_name": "DefaultModelWorker_call.generate_stream_func", "start_time": "2024-10-22 08:37:57.053", "end_time": "2024-10-22 08:37:58.301", "metadata": {"output": "您可能想询问的是“隐蔽工程预验收/验收单的验收日期是什么时候”。根据提供的信息，验收日期是2023年1月13日。"}}
{"span_type": "base", "trace_id": "6b8c96d4d9695647cd030fc50aedda78", "span_id": "6b8c96d4d9695647cd030fc50aedda78:f2990c6519cf6f39", "parent_span_id": "6b8c96d4d9695647cd030fc50aedda78:90ac957d284c07b0", "operation_name": "DefaultModelWorker.async_generate_stream", "start_time": "2024-10-22 08:37:57.052", "end_time": "2024-10-22 08:37:58.301", "metadata": null}
{"span_type": "base", "trace_id": "6b8c96d4d9695647cd030fc50aedda78", "span_id": "6b8c96d4d9695647cd030fc50aedda78:90ac957d284c07b0", "parent_span_id": "6b8c96d4d9695647cd030fc50aedda78:04508085a61a5cae", "operation_name": "WorkerManager.generate_stream", "start_time": "2024-10-22 08:37:57.052", "end_time": "2024-10-22 08:37:58.301", "metadata": null}
{"span_type": "base", "trace_id": "6b8c96d4d9695647cd030fc50aedda78", "span_id": "6b8c96d4d9695647cd030fc50aedda78:9a954dc22eac2e9f", "parent_span_id": "6b8c96d4d9695647cd030fc50aedda78:f1df8a405e88a879", "operation_name": "dbgpt.awel.operator.call_stream.iterate", "start_time": "2024-10-22 08:37:57.048", "end_time": "2024-10-22 08:37:58.301", "metadata": null}
{"span_type": "base", "trace_id": "6b8c96d4d9695647cd030fc50aedda78", "span_id": "6b8c96d4d9695647cd030fc50aedda78:04508085a61a5cae", "parent_span_id": "6b8c96d4d9695647cd030fc50aedda78:d8d4eedfe70df046", "operation_name": "BaseChat.stream_call", "start_time": "2024-10-22 08:37:57.045", "end_time": "2024-10-22 08:37:58.311", "metadata": {"model": "proxyllm", "messages": [{"role": "system", "content": " Based on the known information below, provide users with professional and concise answers to their questions.\nconstraints:\n 1.Ensure to include original markdown formatting elements such as images, links, tables, or code blocks without alteration in the response if they are present in the provided information.\n For example, image format should be
, link format xxx, table format should be represented with |xxx|xxx|xxx|, and code format with xxx.\n 2.If the information available in the knowledge base is insufficient to answer the question, state clearly: \"The content provided in the knowledge base is not enough to answer this question,\" and avoid making up answers.\n 3.When responding, it is best to summarize the points in the order of 1, 2, 3, And displayed in markdwon format.\n known information: \n 山真国际\n question:\n 在么,when answering, use the same language as the \"user\".\n", "round_index": 0}, {"role": "human", "content": "在么", "round_index": 0}], "temperature": 0.6, "max_new_tokens": 8192, "echo": false, "span_id": "6b8c96d4d9695647cd030fc50aedda78:d8d4eedfe70df046", "context": {"stream": true, "cache_enable": false, "user_name": "1", "sys_code": "1840310408651038721", "span_id": "6b8c96d4d9695647cd030fc50aedda78:d8d4eedfe70df046", "chat_mode": "chat_knowledge", "extra": {}}}}
{"span_type": "base", "trace_id": "6b8c96d4d9695647cd030fc50aedda78", "span_id": "6b8c96d4d9695647cd030fc50aedda78:f88ca5881ba62ffc", "parent_span_id": "6b8c96d4d9695647cd030fc50aedda78:d8d4eedfe70df046", "operation_name": "stream_generator", "start_time": "2024-10-22 08:37:56.659", "end_time": "2024-10-22 08:37:58.388", "metadata": null}
{"span_type": "base", "trace_id": "fda8f5b76acf8db27b9e09bd72e8df3a", "span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:f550123037e5b8d0", "parent_span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:b783f0131d1e4a06", "operation_name": "dbgpt.awel.operator.call_stream.iterate", "start_time": "2024-10-23 10:00:54.412", "end_time": null, "metadata": null}
{"span_type": "base", "trace_id": "fda8f5b76acf8db27b9e09bd72e8df3a", "span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:90977e69cdcd00e5", "parent_span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:f550123037e5b8d0", "operation_name": "WorkerManager.get_model_metadata", "start_time": "2024-10-23 10:00:54.412", "end_time": null, "metadata": null}
{"span_type": "base", "trace_id": "fda8f5b76acf8db27b9e09bd72e8df3a", "span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:90977e69cdcd00e5", "parent_span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:f550123037e5b8d0", "operation_name": "WorkerManager.get_model_metadata", "start_time": "2024-10-23 10:00:54.412", "end_time": "2024-10-23 10:00:54.415", "metadata": null}
{"span_type": "base", "trace_id": "fda8f5b76acf8db27b9e09bd72e8df3a", "span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:4e7e0f997fa60450", "parent_span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:0fab4e7e7033c32d", "operation_name": "WorkerManager.generate_stream", "start_time": "2024-10-23 10:00:54.416", "end_time": null, "metadata": null}
{"span_type": "base", "trace_id": "fda8f5b76acf8db27b9e09bd72e8df3a", "span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:74d69849433fd15b", "parent_span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:4e7e0f997fa60450", "operation_name": "DefaultModelWorker.async_generate_stream", "start_time": "2024-10-23 10:00:54.416", "end_time": null, "metadata": null}
{"span_type": "base", "trace_id": "fda8f5b76acf8db27b9e09bd72e8df3a", "span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:1d47f34730ede03e", "parent_span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:74d69849433fd15b", "operation_name": "DefaultModelWorker_call.generate_stream_func", "start_time": "2024-10-23 10:00:54.417", "end_time": null, "metadata": {"is_async_func": true, "llm_adapter": ", link format xxx, table format should be represented with |xxx|xxx|xxx|, and code format with xxx.\n 2.If the information available in the knowledge base is insufficient to answer the question, state clearly: \"The content provided in the knowledge base is not enough to answer this question,\" and avoid making up answers.\n 3.When responding, it is best to summarize the points in the order of 1, 2, 3, And displayed in markdwon format.\n known information: \n 此外，\n question:\n 造纸工艺,when answering, use the same language as the \"user\".\n", "round_index": 0}, {"role": "human", "content": "造纸工艺", "round_index": 0}], "temperature": 0.6, "max_new_tokens": 8192, "echo": false, "span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:4e7e0f997fa60450", "context": {"stream": true, "cache_enable": false, "user_name": "1815928923353604097", "sys_code": "1840310408651038721", "span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:6b91ef5d6400d353", "chat_mode": "chat_knowledge", "extra": {}}, "convert_to_compatible_format": false, "string_prompt": "system: Based on the known information below, provide users with professional and concise answers to their questions.\nconstraints:\n 1.Ensure to include original markdown formatting elements such as images, links, tables, or code blocks without alteration in the response if they are present in the provided information.\n For example, image format should be
, link format xxx, table format should be represented with |xxx|xxx|xxx|, and code format with xxx.\n 2.If the information available in the knowledge base is insufficient to answer the question, state clearly: \"The content provided in the knowledge base is not enough to answer this question,\" and avoid making up answers.\n 3.When responding, it is best to summarize the points in the order of 1, 2, 3, And displayed in markdwon format.\n known information: \n 此外，\n question:\n 造纸工艺,when answering, use the same language as the \"user\".\n\nhuman: 造纸工艺", "prompt_echo_len_char": -1, "has_format_prompt": false, "prompt": "system: Based on the known information below, provide users with professional and concise answers to their questions.\nconstraints:\n 1.Ensure to include original markdown formatting elements such as images, links, tables, or code blocks without alteration in the response if they are present in the provided information.\n For example, image format should be
, link format xxx, table format should be represented with |xxx|xxx|xxx|, and code format with xxx.\n 2.If the information available in the knowledge base is insufficient to answer the question, state clearly: \"The content provided in the knowledge base is not enough to answer this question,\" and avoid making up answers.\n 3.When responding, it is best to summarize the points in the order of 1, 2, 3, And displayed in markdwon format.\n known information: \n 此外，\n question:\n 造纸工艺,when answering, use the same language as the \"user\".\n\nhuman: 造纸工艺"}}
{"span_type": "base", "trace_id": "fda8f5b76acf8db27b9e09bd72e8df3a", "span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:f550123037e5b8d0", "parent_span_id": "fda8f5b76acf8db27b9e09bd72e8df3a:b783f0131d1e4a06", "operation_name": "dbgpt.awel.operator.call_stream.iterate", "start_time": "2024-10-23 10:00:54.412", "end_time": "2024-10-23 10:00:54.673", "metadata": null}
"operation_name": "DefaultModelWorker_call.generate_stream_func"缺少end的打印
jlcbj
commented
3 weeks ago 2024-10-25 16:07:14 | INFO | dbgpt.model.proxy.llms.chatgpt | generate_stream_v1 payload:{'stream': True, 'model': 'qwen2', 'temperature': 0.6, 'max_tokens': 8192} 2024-10-25 16:07:14 | INFO | dbgpt.model.proxy.llms.chatgpt | r:ChatCompletionChunk(id='chat-5e7eb88298ae40b18b6921bf03c29a2b', choices=[Choice(delta=ChoiceDelta(content='', function_call=None, refusal=None, role='assistant', tool_calls=None), finish_reason=None, index=0, logprobs=None)], created=1729843634, model='qwen2', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=None) 2024-10-25 16:07:14 | INFO | dbgpt.model.cluster.worker.default_worker | _handle_output incremental_output= 2024-10-25 16:07:14 | INFO | dbgpt.model.cluster.worker.default_worker | is_first_generate, usage: None 2024-10-25 16:07:14 | INFO | dbgpt.model.proxy.llms.chatgpt | r:ChatCompletionChunk(id='chat-5e7eb88298ae40b18b6921bf03c29a2b', choices=[Choice(delta=ChoiceDelta(content='1', function_call=None, refusal=None, role=None, tool_calls=None), finish_reason=None, index=0, logprobs=None)], created=1729843634, model='qwen2', object='chat.completion.chunk', service_tier=None, system_fingerprint=None, usage=None) 2024-10-25 16:07:14 | INFO | dbgpt.model.cluster.worker.default_worker | _handle_output incremental_output=1 加了打印，问题是LLM回答时异常，回答了个1之后就没有后续了，导致dbgpt测卡住。 具体原因是openai client包的问题还是远端vllm openai接口问题待排除。 自问自答后关闭
Search before asking
Operating system information
Linux
Python version information
DB-GPT version
main
Related scenes
Installation Information
[X] Installation From Source
[ ] Docker Installation
[ ] Docker Compose Installation
[ ] Cluster Installation
[ ] AutoDL Image
[ ] Other
Device information
CPU服务器跑主要代码 GPU服务器跑大模型 通过PROXY LLM来通信
Models information
和模型无关，qwen2.5部署的vllm api
What happened
经常遇到问答无响应，只能重启服务来解决
2024-10-15 14:02:09 | INFO | dbgpt.component | chat_completions:chat_knowledge,C_1737362441124798466,proxyllm 2024-10-15 14:02:09 | INFO | dbgpt.component | get_chat_instance:model='proxyllm' messages='你的名字' stream=True temperature=0.2 max_new_tokens=None conv_uid='fb27f614-8aba-11ef-950e-0242ac820902' span_id=None chat_mode='chat_knowledge' chat_param='C_1737362441124798466' user_name='1' sys_code='1737362441124798466' incremental=True enable_vis=True 2024-10-15 14:02:09 | INFO | dbgpt.serve.rag.connector | VectorStore:<class 'dbgpt.storage.vector_store.milvus_store.MilvusStore'> 2024-10-15 14:02:09 | WARNING | dbgpt.storage.vector_store.milvus_store | No relevant docs were retrieved using the relevance score threshold 0.3 2024-10-15 14:02:09 | INFO | dbgpt.core.awel.runner.local_runner | Begin run workflow from end operator, id: 67622b70-cd71-4824-bfaa-22e483d0ea2a, runner: <dbgpt.core.awel.runner.local_runner.DefaultWorkflowRunner object at 0x7f7b9ed7a860> 2024-10-15 14:02:09 | INFO | dbgpt.core.awel.runner.local_runner | Begin run workflow from end operator, id: 4aa255f4-3eca-4221-b26f-e74d1688bfd1, runner: <dbgpt.core.awel.runner.local_runner.DefaultWorkflowRunner object at 0x7f7b9ed7a860> 2024-10-15 14:02:09 | INFO | dbgpt.app.scene.base_chat | payload request: ModelRequest(model='proxyllm', messages=[ModelMessage(role='system', content=' Based on the known information below, provide users with professional and concise answers to their questions.\nconstraints:\n 1.Ensure to include original markdown formatting elements such as images, links, tables, or code blocks without alteration in the response if they are present in the provided information.\n For example, image format should be
, link format xxx, table format should be represented with |xxx|xxx|xxx|, and code format with xxx.\n 2.If the information available in the knowledge base is insufficient to answer the question, state clearly: "The content provided in the knowledge base is not enough to answer this question," and avoid making up answers.\n 3.When responding, it is best to summarize the points in the order of 1, 2, 3, And displayed in markdwon format.\n known information: \n no relevant docs to retrieve\n question:\n 你的名字,when answering, use the same language as the "user".\n', round_index=0), ModelMessage(role='human', content='你的名字', round_index=0)], temperature=0.6, top_p=None, max_new_tokens=8192, stop=None, stop_token_ids=None, context_len=None, echo=False, span_id='1627a87b780982bc2b0c3b9ca7ce3dbd:229a23e3ba36aba8', context=ModelRequestContext(stream=True, cache_enable=False, user_name='1', sys_code='1737362441124798466', conv_uid=None, span_id='1627a87b780982bc2b0c3b9ca7ce3dbd:229a23e3ba36aba8', chat_mode='chat_knowledge', chat_param=None, extra={}, request_id=None))
2024-10-15 14:02:09 | INFO | dbgpt.core.awel.runner.local_runner | Begin run workflow from end operator, id: 13da370e-684a-4205-9edf-8e3ce8eec027, runner: <dbgpt.core.awel.runner.local_runner.DefaultWorkflowRunner object at 0x7f7b9ed7a860>
2024-10-15 14:02:09 | INFO | dbgpt.core.awel.operators.common_operator | branch_input_ctxs 0 result None, is_empty: False
2024-10-15 14:02:09 | INFO | dbgpt.core.awel.operators.common_operator | Skip node name llm_model_cache_node
2024-10-15 14:02:09 | INFO | dbgpt.core.awel.operators.common_operator | branch_input_ctxs 1 result True, is_empty: False
2024-10-15 14:02:09 | INFO | dbgpt.core.awel.runner.local_runner | Skip node name llm_model_cache_node, node id f1fba8c9-b11e-4be8-ab15-f8592f5c0cdd
就没有后续了，HTTP卡住，无response。 此时所有的问答全部不行，不光是知识库，怀疑AWEL底层BUG
What you expected to happen
不能进入系统不可用状态
How to reproduce
运行一段时间就出现该问题，遇到很多次，每次只能重启服务
Additional context
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:8058797a78d304f3", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:229a23e3ba36aba8", "operation_name": "BaseChat.stream_call", "start_time": "2024-10-15 14:02:09.871", "end_time": null, "metadata": {"model": "proxyllm", "messages": [{"role": "system", "content": " Based on the known information below, provide users with professional and concise answers to their questions.\nconstraints:\n 1.Ensure to include original markdown formatting elements such as images, links, tables, or code blocks without alteration in the response if they are present in the provided information.\n For example, image format should be
, link format xxx, table format should be represented with |xxx|xxx|xxx|, and code format with xxx.\n 2.If the information available in the knowledge base is insufficient to answer the question, state clearly: \"The content provided in the knowledge base is not enough to answer this question,\" and avoid making up answers.\n 3.When responding, it is best to summarize the points in the order of 1, 2, 3, And displayed in markdwon format.\n known information: \n no relevant docs to retrieve\n question:\n 你的名字,when answering, use the same language as the \"user\".\n", "round_index": 0}, {"role": "human", "content": "你的名字", "round_index": 0}], "temperature": 0.6, "max_new_tokens": 8192, "echo": false, "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:229a23e3ba36aba8", "context": {"stream": true, "cache_enable": false, "user_name": "1", "sys_code": "1737362441124798466", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:229a23e3ba36aba8", "chat_mode": "chat_knowledge", "extra": {}}}}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:df68e5e3ed71bd10", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:8058797a78d304f3", "operation_name": "dbgpt.awel.operator.call_stream", "start_time": "2024-10-15 14:02:09.872", "end_time": null, "metadata": null}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:2f871b60700761d4", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:df68e5e3ed71bd10", "operation_name": "dbgpt.awel.workflow.run_workflow", "start_time": "2024-10-15 14:02:09.872", "end_time": null, "metadata": {"exist_dag_ctx": false, "event_loop_task_id": 140160217188448, "streaming_call": true, "awel_node_id": "13da370e-684a-4205-9edf-8e3ce8eec027"}}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:f1cf43c91de01f68", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:2f871b60700761d4", "operation_name": "dbgpt.awel.workflow.run_operator", "start_time": "2024-10-15 14:02:09.872", "end_time": null, "metadata": {"awel_node_id": "e39a7afd-004d-4c8f-8fae-b1f142dcfd48", "awel_node_type": "InputOperator(node_id=e39a7afd-004d-4c8f-8fae-b1f142dcfd48)", "state": "running", "task_log_id": "e39a7afd-004d-4c8f-8fae-b1f142dcfd48_1"}}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:f1cf43c91de01f68", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:2f871b60700761d4", "operation_name": "dbgpt.awel.workflow.run_operator", "start_time": "2024-10-15 14:02:09.872", "end_time": "2024-10-15 14:02:09.873", "metadata": {"awel_node_id": "e39a7afd-004d-4c8f-8fae-b1f142dcfd48", "awel_node_type": "InputOperator(node_id=e39a7afd-004d-4c8f-8fae-b1f142dcfd48)", "state": "success", "task_log_id": "e39a7afd-004d-4c8f-8fae-b1f142dcfd48_1", "skip_node_ids": ""}}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:4b5fedf5dcd5dac5", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:2f871b60700761d4", "operation_name": "dbgpt.awel.workflow.run_operator", "start_time": "2024-10-15 14:02:09.873", "end_time": null, "metadata": {"awel_node_id": "e23fc041-f307-43fd-80c8-0fce8e545a47", "awel_node_type": "ModelCacheBranchOperator(node_id=e23fc041-f307-43fd-80c8-0fce8e545a47)", "state": "running", "task_log_id": "e23fc041-f307-43fd-80c8-0fce8e545a47_1"}}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:4b5fedf5dcd5dac5", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:2f871b60700761d4", "operation_name": "dbgpt.awel.workflow.run_operator", "start_time": "2024-10-15 14:02:09.873", "end_time": "2024-10-15 14:02:09.874", "metadata": {"awel_node_id": "e23fc041-f307-43fd-80c8-0fce8e545a47", "awel_node_type": "ModelCacheBranchOperator(node_id=e23fc041-f307-43fd-80c8-0fce8e545a47)", "state": "success", "task_log_id": "e23fc041-f307-43fd-80c8-0fce8e545a47_1", "skip_node_ids": ""}}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:5e7b42789e9006ff", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:2f871b60700761d4", "operation_name": "dbgpt.awel.workflow.run_operator", "start_time": "2024-10-15 14:02:09.875", "end_time": "2024-10-15 14:02:09.875", "metadata": {"awel_node_id": "678bd746-23f7-4b02-a966-6c5354ae5643", "awel_node_type": "ModelStreamSaveCacheOperator(node_id=678bd746-23f7-4b02-a966-6c5354ae5643)", "state": "success", "task_log_id": "678bd746-23f7-4b02-a966-6c5354ae5643_1", "skip_node_ids": "f1fba8c9-b11e-4be8-ab15-f8592f5c0cdd"}}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:81e7aed4b654b4f7", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:2f871b60700761d4", "operation_name": "dbgpt.awel.workflow.run_operator", "start_time": "2024-10-15 14:02:09.875", "end_time": null, "metadata": {"awel_node_id": "13da370e-684a-4205-9edf-8e3ce8eec027", "awel_node_type": "BranchJoinOperator(node_id=13da370e-684a-4205-9edf-8e3ce8eec027)", "state": "running", "task_log_id": "13da370e-684a-4205-9edf-8e3ce8eec027_1"}}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:81e7aed4b654b4f7", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:2f871b60700761d4", "operation_name": "dbgpt.awel.workflow.run_operator", "start_time": "2024-10-15 14:02:09.875", "end_time": "2024-10-15 14:02:09.875", "metadata": {"awel_node_id": "13da370e-684a-4205-9edf-8e3ce8eec027", "awel_node_type": "BranchJoinOperator(node_id=13da370e-684a-4205-9edf-8e3ce8eec027)", "state": "success", "task_log_id": "13da370e-684a-4205-9edf-8e3ce8eec027_1", "skip_node_ids": "f1fba8c9-b11e-4be8-ab15-f8592f5c0cdd"}}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:2f871b60700761d4", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:df68e5e3ed71bd10", "operation_name": "dbgpt.awel.workflow.run_workflow", "start_time": "2024-10-15 14:02:09.872", "end_time": "2024-10-15 14:02:09.875", "metadata": {"exist_dag_ctx": false, "event_loop_task_id": 140160217188448, "streaming_call": true, "awel_node_id": "13da370e-684a-4205-9edf-8e3ce8eec027"}}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:df68e5e3ed71bd10", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:8058797a78d304f3", "operation_name": "dbgpt.awel.operator.call_stream", "start_time": "2024-10-15 14:02:09.872", "end_time": "2024-10-15 14:02:09.875", "metadata": null}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:eb15953f8a3d81dc", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:df68e5e3ed71bd10", "operation_name": "dbgpt.awel.operator.call_stream.iterate", "start_time": "2024-10-15 14:02:09.875", "end_time": null, "metadata": null}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:a1f293c834254b44", "parent
_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:eb15953f8a3d81dc", "operation_name": "WorkerManager.get_model_metadata", "start_time": "2024-10-15 14:02:09.877", "end_time": null, "metadata": null}
{"span_type": "base", "trace_id": "4ea6178a1eeee89dbd15b0f42096d8a3", "span_id": "4ea6178a1eeee89dbd15b0f42096d8a3:aa085302143542b6", "parent_span_id": null, "operation_name": "DB-GPT-Webserver", "start_time": "2024-10-15 14:03:04.458", "end_time": null, "metadata": {"path": "/api/v1/chat/dialogue/list"}}
{"span_type": "base", "trace_id": "4ea6178a1eeee89dbd15b0f42096d8a3", "span_id": "4ea6178a1eeee89dbd15b0f42096d8a3:aa085302143542b6", "parent_span_id": null, "operation_name": "DB-GPT-Webserver", "start_time": "2024-10-15 14:03:04.458", "end_time": "2024-10-15 14:03:04.487", "metadata": {"path": "/api/v1/chat/dialogue/list"}}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:a1f293c834254b44", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:eb15953f8a3d81dc", "operation_name": "WorkerManager.get_model_metadata", "start_time": "2024-10-15 14:02:09.877", "end_time": "2024-10-15 14:03:09.715", "metadata": null}
{"span_type": "base", "trace_id": "1627a87b780982bc2b0c3b9ca7ce3dbd", "span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:eb15953f8a3d81dc", "parent_span_id": "1627a87b780982bc2b0c3b9ca7ce3dbd:df68e5e3ed71bd10", "operation_name": "dbgpt.awel.operator.call_stream.iterate", "start_time": "2024-10-15 14:02:09.875", "end_time": "2024-10-15 14:03:09.715", "metadata": null}
Are you willing to submit PR?