AUTOGEN issue TypeError: Completions.create()

[AvATech010]

Hi all.

Having a problem after installing autogen. It starts then blows up after user proxy....Anyone can point me in the right direction to fix this? TypeError: Completions.create() got an unexpected keyword argument 'request_timeout' I already have the timeout to 600

Thanks!

app.py

user_proxy (to CTO):

Write python code to output numbers 1 to 100, and then store the code in a file

---

Traceback (most recent call last): File "/Users/ronayne/AIprojects/autogen/autogen/app.py", line 38, in user_proxy.initiate_chat( File "/Users/ronayne/AIprojects/autogen/autogen/autogen/agentchat/conversable_agent.py", line 550, in initiate_chat self.send(self.generate_init_message(context), recipient, silent=silent) File "/Users/ronayne/AIprojects/autogen/autogen/autogen/agentchat/conversable_agent.py", line 348, in send recipient.receive(message, self, request_reply, silent) File "/Users/ronayne/AIprojects/autogen/autogen/autogen/agentchat/conversable_agent.py", line 481, in receive reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/ronayne/AIprojects/autogen/autogen/autogen/agentchat/conversable_agent.py", line 940, in generate_reply final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/ronayne/AIprojects/autogen/autogen/autogen/agentchat/conversable_agent.py", line 625, in generate_oai_reply response = client.create( ^^^^^^^^^^^^^^ File "/Users/ronayne/AIprojects/autogen/autogen/autogen/oai/client.py", line 250, in create response = self._completions_create(client, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/ronayne/AIprojects/autogen/autogen/autogen/oai/client.py", line 332, in _completions_create response = completions.create(params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/ronayne/miniconda3/lib/python3.11/site-packages/openai/_utils/_utils.py", line 301, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ TypeError: Completions.create() got an unexpected keyword argument 'request_timeout'

[rickyloynd-microsoft]

It looks like you may not be using the latest version pyautogen==0.2.0. Can you do pip freeze to see your pyautogen and openai versions?

[NewMountain]

Ditto here as well:

(textgen) yeti@yeti:~/code/ml/local_llm/autogen_experiment$ python app.py 
user_proxy (to assistant):

Give me a python script that solves the classic "FIZZBUZZ" problem and save the file
as fizzbuzz.py.

--------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/yeti/code/ml/local_llm/autogen_experiment/app.py", line 40, in <module>
    user_proxy.initiate_chat(assistant, message=task)
  File "/home/yeti/anaconda3/envs/textgen/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 550, in initiate_chat
    self.send(self.generate_init_message(**context), recipient, silent=silent)
  File "/home/yeti/anaconda3/envs/textgen/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 348, in send
    recipient.receive(message, self, request_reply, silent)
  File "/home/yeti/anaconda3/envs/textgen/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 481, in receive
    reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yeti/anaconda3/envs/textgen/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 906, in generate_reply
    final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yeti/anaconda3/envs/textgen/lib/python3.11/site-packages/autogen/agentchat/conversable_agent.py", line 625, in generate_oai_reply
    response = client.create(
               ^^^^^^^^^^^^^^
  File "/home/yeti/anaconda3/envs/textgen/lib/python3.11/site-packages/autogen/oai/client.py", line 247, in create
    response = self._completions_create(client, params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yeti/anaconda3/envs/textgen/lib/python3.11/site-packages/autogen/oai/client.py", line 327, in _completions_create
    response = completions.create(**params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yeti/anaconda3/envs/textgen/lib/python3.11/site-packages/openai/_utils/_utils.py", line 301, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
TypeError: Completions.create() got an unexpected keyword argument 'request_timeout'

Running pip freeze | grep -i pyauto, I get pyautogen==0.2.0.

Running Ubuntu 22.04 in a conda environment (py 3.11).

[rickyloynd-microsoft]

request_timeout went away with pyautogen==0.2.0 and openai>=1.0.0. To run code that you wrote earlier, you'll want to review the migration guide here.

[NewMountain]

Thank you @rickyloynd-microsoft . I appreciate the fast response.

For anyone in the future reading this that might have wound up here from a similar place. I was working through the tutorial Use AutoGen with ANY Open-Source Model! (RunPod + TextGen WebUI)

The steps I ran to get here (roughly following along with the tutorial):

conda create -n textgen python=3.11
conda activate textgen
cd ~/code/ml
mkdir local_llm
cd local_llm

conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui
pip install -r requirements.txt

cd ~/code/ml/local_llm/text-generation-webui
python server.py

Click Session Tab: Click OpenAI checkbox and then click "Apply flags/extensions and restart" button

I dropped a print("PARAMS:", params) in autogen/oai/client.py and can definitely see the request_timeout argument in there.

{'messages': [{'content': 'You are a coder specializing in Python.', 'role': 'system'}, {'content': '\nGive me a python script that solves the classic "FIZZBUZZ" problem and save the file\nas fizzbuzz.py.\n', 'role': 'user'}], 'model': 'gpt-4', 'request_timeout': 600, 'seed': 42, 'temperature': 0, 'api_type': 'open_ai', 'api_base': 'http://127.0.0.1:5001/v1', 'stream': False}

Here are the steps I took to get myself unstuck from that tutorial (written for others who may find themselves in the same situation)

1. In text-generation-webui, run pip install -r extensions/openai/requirements.txt as outlined in the text-generation-webui docs
2. Restart the server at the command line python server.py. If you do it this way, you will need to re-load your model and re-enable the openai api in the session tab. Alternatively, if you use the command python server.py --api, you will automatically have the api enabled on port 500.
3. Validate the API is now working with a curl request (you should expect a json response that makes sense)

   curl http://127.0.0.1:5000/v1/completions \
   -H "Content-Type: application/json" \
   -d '{
   "prompt": "This is a cake recipe:\n\n1.",
   "max_tokens": 200,
   "temperature": 1,
   "top_p": 0.9,
   "seed": 10
   }'

4. Update your app.py to this (instead of what was instructed in the Youtube tutorial)

import autogen

config_list = [
    {
        # Now an invalid argument, not sure what the right argument here is
        #"api_type": "open_ai",
        # api_base is now base_url
        "base_url": "http://127.0.0.1:5000/v1",
        "api_key": "NULL",
    }
]

llm_config = {
    # Note: This will cause a bug in the latest autogen
    # Do not use request_timeout, it is now `timeout`
    # See: https://microsoft.github.io/autogen/docs/Installation#migration-guide-to-v02
    "timeout": 600,
    "seed": 42,
    "config_list": config_list,
    "temperature": 0,
}

assistant = autogen.AssistantAgent(
    name="assistant",
    system_message="You are a coder specializing in Python.",
    llm_config=llm_config,
)

user_proxy = autogen.UserProxyAgent(
    # Note: We want to make sure the program is done using the TERMINATE keyword we define
    name="user_proxy",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=10,
    is_termination_msg=lambda x: x.get("content", "").rstrip().endswith("TERMINATE"),
    code_execution_config={"work_dir": "work"},
    llm_config=llm_config,
    #
    system_message="""Reply TERMINATE if the task has been solved at full satisfaction.
Otherwise, reply CONTINUE, or the reason why the task is not solved yet.""",
)

task = """
Give me a python script that solves the classic "FIZZBUZZ" problem and save the file
as fizzbuzz.py.
"""

user_proxy.initiate_chat(assistant, message=task)

There appears to be an issue with the assistant not getting a response, but at least everything is working without throwing at this point.

[rickyloynd-microsoft]

Thanks @NewMountain for the updated app.py! I'll resolve this issue, but please open it or another one for any further problems.

[dyoungjr08]

Please help. I have copied your code but my chat shows up with blank responses. ```

python main.py
user_proxy (to assistant):

Write me the python code to print the numbers 1 thru 10

---

assistant (to user_proxy):

---

user_proxy (to assistant):

---

assistant (to user_proxy):

---

My test response looks like:

user_proxy (to assistant):

Write me the python code to print the numbers 1 thru 10

--------------------------------------------------------------------------------
assistant (to user_proxy):

--------------------------------------------------------------------------------
user_proxy (to assistant):

--------------------------------------------------------------------------------
assistant (to user_proxy):

--------------------------------------------------------------------------------

curl https://-5000.proxy.runpod.net/v1/chat/completions >
-H "Content-Type: application/json" \ -d '{ "messages": [{"role": "user", "content": "Hello!" }], "mode": "instruct", "instruction_template": "Alpaca"}'

{"id":"chatcmpl-1702106134919602432","object":"chat.completions","created":1702106134,"model":"ehartford_dolphin-2.1-mistral-7b","choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"\nHello there!\n\nFirstly, I wanted to apologize for any misunderstandings or delays. We understand the importance of efficient customer service and we are working on improving our process to serve you better.\n\nWith respect to your task, I would be happy to assist you. Start by outlining the information you would like to receive. It will help us provide you with the right details.\n\nPlease feel free to share any concerns, issues, or suggestions you may have. Your feedback is essential to us to keep improving.\n\nLet's work towards a resolution that will leave you fully satisfied with our service.\n\nSincerely,\n[Your Name]\n[Your Position]\n[Business Name]\n[Phone Number]\n[Email Address]"}}],"usage":{"prompt_tokens":36,"completion_tokens":166,"total_tokens":202}}(textgen) root@cc17716ee04a:/workspace/text-generation-webui#

However when I do a test to the openAI endpoint the json appear with the proper structure. Is that the issue?

curl https://api.openai.com/v1/chat/completions   
-H "Content-Type: application/json"   
-H "Authorization: Bearer $OPENAI_API_KEY"   
-d '{  "model": "gpt-3.5-turbo",
     "messages": [{"role": "user", "content": "Write the python code to display numbers 1 thru 10"}],
     "temperature": 0.7  }'
{
  "id": "chatcmpl-8Tm2l3CMKf5UFSc64uswvYqyEgWW6",
  "object": "chat.completion",
  "created": 1702106883,
  "model": "gpt-3.5-turbo-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Here's the Python code to display numbers 1 through 10:\n\n```python\nfor num in range(1, 11):\n    print(num)\n```\n\nThis code uses a `for` loop to iterate over the numbers in the range from 1 to 10 (inclusive), and then prints each number on a separate line."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 19,
    "completion_tokens": 68,
    "total_tokens": 87
  },
  "system_fingerprint": null
}

[rickyloynd-microsoft]

@dyoungjr08 Are you able to run a simple code example from the autogen repo, like the one on the README page?

[dyoungjr08]

@dyoungjr08 Are you able to run a simple code example from the autogen repo, like the one on the README page?

Yes. I can run this for example: https://colab.research.google.com/github/microsoft/autogen/blob/main/notebook/agentchat_auto_feedback_from_code_execution.ipynb#scrollTo=sxnbrnw2Tf3f

I can run it colab and visual studio. But when I switch to the textgen-webui openai api it just returns blank responses. I even tried using "google" as a base_url just to make sure the url was updating and that it would crash if an invalid response was received. It crashes as it should. I feel that it has something to do with the JSON structure but I could be wrong. It just seems as though it is receiving a response of some sort because the script does not crash. Thank you for your help.

[rickyloynd-microsoft]

I see some discussion of textgen on our Discord channel, so you may want to ask this question there.

[matsuobasho]

@NewMountain thanks for that guidance, it really helped. I just followed from point 4 though, and it worked fine. Question is - is there a way to run a server with multiple models with LM Studio so that different assistants can leverage different LMs?

[johnwayne087]

If anyone else ends up here trying to use the new AutoGen Studio with OobaBooga Text Gen Webui, I just hardcoded the base url in /home/user/autogen/samples/apps/autogen-studio/venv/lib/python3.10/site-packages/autogenstudio/utils/utils.py.

Line 308 llm_config = LLMConfig( base_url="127.0.0.1:8000/v1/", config_list=[{"model": "gpt-4"}], temperature=0, )

[rickyloynd-microsoft]

@NewMountain thanks for that guidance, it really helped. I just followed from point 4 though, and it worked fine. Question is - is there a way to run a server with multiple models with LM Studio so that different assistants can leverage different LMs?

Autogen has no problem mapping different LLMs to different agents, but I don't know if LM Studio can handle that.