cg和ltp的std都出现了问题：Error: Worker not responding

wangyanli3630 commented 8 months ago

cg和ltp的std都出现了问题：Error: Worker not responding，且没有一条数据执行成功

zhc7 commented 8 months ago

ltp出现worker not responding是因为ltp内部需要调openai的api（用turbo做海龟汤的裁判），所以如果没有正确设置openai的key或者连接不稳定或者并发过高等都有可能导致问题。关于cg可以分享一点更具体的信息吗？如启动的worker数量、评测的模型的情况等。

wangyanli3630 commented 8 months ago

我的配置如下，麻烦大佬帮忙看下：

openai-chat.yaml

module: src.client.agents.HTTPAgent
parameters:
  url: https://api.openai.com/v1/chat/completions
  headers:
    Content-Type: application/json
    Authorization: Bearer sk-___________
  body:
    temperature: 0
  prompter:
    name: role_content_dict
    args:
      agent_role: assistant
  return_format: "{response[choices][0][message][content]}"

default.yaml

import: definition.yaml

concurrency:
  task:
    # dbbench-std: 1
    # os-std: 1
    # alfworld-std: 1
    # webshop-std: 1
    # m2w-std: 1
    ltp-std: 1
    # kg-std: 1
    cg-std: 1

  agent:
    gpt-3.5-turbo-0613: 5

assignments: # List[Assignment] | Assignment
  - agent: # "task": List[str] | str ,  "agent": List[str] | str
      - gpt-3.5-turbo-0613
    task:
      # - dbbench-std
      # - os-std
      # - alfworld-std
      # - webshop-std
      # - m2w-std
      - ltp-std
      # - kg-std
      - cg-std

output: "outputs/{TIMESTAMP}"

start_task.yaml

definition:
  import: tasks/task_assembly.yaml

start:
  # dbbench-std: 1
  # os-std: 1
  # alfworld-std: 1
  # webshop-std: 1
  # m2w-std: 1
  ltp-std: 1
  # kg-std: 1
  cg-std: 1

Longin-Yu commented 8 months ago

Is your OpenAI key a free trial key? If that's the case, it's important to be aware that free trial keys have rate limits that can be reached quite easily. You can find more information about these rate limits in the OpenAI Documentation. Additionally, the process of evaluating and making LTP calls to the OpenAI API (specifically using the gpt-3.5-turbo model) may lead to faster consumption of your key's usage capacity.

Ludison commented 8 months ago

@nov3630 同样出现了这个报错，请问有解决这个问题吗。我这边的区别是没有使用gpt3.5，使用的开源模型

GhostXu11 commented 7 months ago

@Ludison 你问题解决了么我也是开源模型遇到了这个问题而且我在openai-chat.yaml中已经设置了key

alg-bug-engineer commented 6 months ago

+1，同样的错误

Taishi-N324 commented 4 months ago

Has anyone solved this problem? I have the same problem.

THUDM / AgentBench