THUDM / AgentBench

A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)
https://llmbench.ai
Apache License 2.0
2.01k stars 136 forks source link

[Bug/Assistance] 怎么配置configs/agents/openai-chat.yaml #102

Closed yananchen1989 closed 6 months ago

yananchen1989 commented 6 months ago

你好,目前在openai-chat.yaml 中,我是这么配置的,其中fa7639a52b6f421dab015 是我的key。 不知道这样对不对,我在执行 python -m src.client.agent_test --config configs/agents/api_agents.yaml --agent gpt-3.5-turbo-0613之后一直失败:

module: src.client.agents.HTTPAgent
parameters:
  url: https://api.openai.com/v1/chat/completions
  headers:
    Content-Type: application/json
    Authorization: Bearer fa7639a52b6f421dab015
  body: 
    temperature: 0
  prompter:
    name: role_content_dict
    args:
      agent_role: assistant
  return_format: "{response[choices][0][message][content]}"

错误信息如下:

{'gpt-3.5-turbo-0613': {'module': 'src.client.agents.HTTPAgent', 'parameters': {'url': 'https://api.openai.com/v1/chat/completions', 'headers': {'Content-Type': 'application/json', 'Authorization': 'Bearer fa7639a52b6f421dab015 '}, 'body': {'temperature': 0, 'model': 'gpt-3.5-turbo-0613', 'max_tokens': 512}, 'prompter': {'name': 'role_content_dict', 'args': {'agent_role': 'assistant'}}, 'return_format': '{response[choices][0][message][content]}', 'name': 'gpt-3.5-turbo-0613'}}, 'text-davinci-003': {'module': 'src.client.agents.HTTPAgent', 'parameters': {'name': 'text-davinci-003', 'url': 'https://api.openai.com/v1/completions', 'headers': {'Content-Type': 'application/json', 'Authorization': 'Bearer fa7639a52b6f421dab015 '}, 'body': {'model': 'text-davinci-003', 'temperature': 0, 'max_tokens': 512}, 'prompter': {'name': 'prompt_string'}, 'return_format': '{response[choices][0][text]}'}}, 'text-davinci-002': {'module': 'src.client.agents.HTTPAgent', 'parameters': {'name': 'text-davinci-002', 'url': 'https://api.openai.com/v1/completions', 'headers': {'Content-Type': 'application/json', 'Authorization': 'Bearer fa7639a52b6f421dab015 '}, 'body': {'model': 'text-davinci-002', 'temperature': 0, 'max_tokens': 512}, 'prompter': {'name': 'prompt_string'}, 'return_format': '{response[choices][0][text]}'}}} ================= USER ===================

What is the capital city of British Columbia, Canada Warning: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) Warning: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) Warning: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) Failed.

yananchen1989 commented 6 months ago

我的key是没有问题的,已经通过

  response = openai.Completion.create(  # type: ignore
      prompt=prompt,
      engine=engine,
      temperature=temperature,
      max_tokens=max_tokens,
      top_p=top_p,
      stop=[stop_token],
  )
  answer: str = response["choices"][0]["text"]

等单元测试方法测试了。

我在agentbench代码里,并未找到类似的对openai的调用。