[Bug]: base_url for local llm is invalid

[hijkzzz]

Describe the bug

For autogen with localllm

import autogen

config_list = [
    {
        "model": "cognitivecomputations/dolphin-2.6-mixtral-8x7b",
        "base_url": "http://10.34.2.53:12345/v1",
        "api_key": "",
        "api_type": "open_ai",
    },
]

llm_config = {"config_list": config_list, "seed": 42}

# The user agent
user_proxy = autogen.UserProxyAgent(
    name="User_proxy",
    system_message="A human admin.",
    code_execution_config={"last_n_messages": 2, "work_dir": "groupchat"},
    human_input_mode="TERMINATE",  # needed?
    is_termination_msg=lambda msg: "TERMINATE" in msg["content"],
    default_auto_reply="You are going to figure all out by your own. "
    "Work by yourself, the user won't reply until you output `TERMINATE` to end the conversation.",
)

# In the AutoGen example, we create an AssistantAgent to play the role of the coder
coder = autogen.AssistantAgent(
    name="Coder",
    llm_config=llm_config,
    system_message=f"I am a 10x engineer, trained in Python. I was the first engineer at Uber. I will wrap my code in ``` ```.",
    human_input_mode="TERMINATE",
)

# Begin the group chat with a message from the user
user_proxy.initiate_chat(
    coder,
    message="Write a Function to print Numbers 1 to 10"
    )

I got the error

  File "/home/scratch.jianh_inf/miniconda3/envs/agent/lib/python3.10/site-packages/h11/_util.py", line 91, in validate
    raise LocalProtocolError(msg)
h11._util.LocalProtocolError: Illegal header value b'Bearer '

... 

    raise APIConnectionError(request=request) from err
openai.APIConnectionError: Connection error.

It works well with previous pyautogen==0.1.14

Steps to reproduce

No response

Expected Behavior

No response

Screenshots and logs

No response

Additional Information

No response

[rickyloynd-microsoft]

@victordibia Any idea here?

[zhaoyilun]

I think you should try "api_base" instead of "base_url"

[hijkzzz]

I think you should try "api_base" instead of "base_url"

It seems api_base is deprecated in new OpenAI client

[Haripritamreddy]

Just add api_key="anything" the error Illegal header value b'Bearer ' mean that you are not sending anything in the header.

[victordibia]

I have not worked with localllm but @Haripritamreddy's answer above seems reasonable. Also see here.

[hijkzzz]

@Haripritamreddy It works. Thanks!