[Issue]: `reflection_with_llm` not working with ollama/llama3

[piratos]

Describe the issue

using

summary_method="reflection_with_llm",

Does not seem to work as the result.summary of the chat session is the last message instead of summary. by enabling openai lib debug I see that the summary request is sent as a message with {"role": "system"}

If I copy the same request (which also contain all the previous messages) and run it manually it indeed returns the last message. If replace {"role": "system"} with {"role": "user"} for the summary request message it works. (which makes sense, to me at least)

Steps to reproduce

from autogen import ConversableAgent

llm_config = {
    "model": "llama3:8b-instruct-fp16", # model
    "base_url" : "http://X.X.X.X:11434/v1", # Ollama openai compatible endpoint
    "api_key": "NULL"
}

config_list = {"config_list": [llm_config]}

student = ConversableAgent(
    "student",
    system_message="You are a student who is willing to learn",
    llm_config=config_list,
    human_input_mode="NEVER",
)

teacher = ConversableAgent(
    "teacher",
    system_message="You are a math teacher",
    llm_config=config_list,
    human_input_mode="NEVER",
)

result = student.initiate_chat(
    teacher,
    message="what is the Pythagorean Trigonometric Identity?",
    summary_method="reflection_with_llm",
    max_turns=2,
)

print(f"Summary of the chat \n{result.summary}")

Screenshots and logs

Result

teacher (to student):

Thank you for your enthusiasm!

As a math teacher, I'm always excited to explore new topics and applications. Since we've covered the Pythagorean trigonometric identity, let's dive deeper into trigonometry! There are many more identities and formulas waiting to be discovered.

Next, I'd like to introduce you to the sum and difference formulas for sine and cosine. These formulas are crucial in solving triangles and have numerous applications in physics, engineering, and computer science.

We can also explore the law of sines and the law of cosines, which allow us to solve triangles using ratios of side lengths or angles. These laws have significant implications in navigation, astronomy, and construction.

If you're ready for a challenge, we could also tackle some advanced trigonometry topics like parametric equations, polar coordinates, and triple integrals.

Or, if you'd prefer a break from math, we could venture into other areas of science or explore real-world applications of trigonometry in fields like medicine, computer graphics, or music!

What sounds appealing to you?

--------------------------------------------------------------------------------
Summary of the chat 
Thank you for your enthusiasm!
As a math teacher, I'm always excited to explore new topics and applications. Since we've covered the Pythagorean trigonometric identity, let's dive deeper into trigonometry! There are many more identities and formulas waiting to be discovered.
Next, I'd like to introduce you to the sum and difference formulas for sine and cosine. These formulas are crucial in solving triangles and have numerous applications in physics, engineering, and computer science.
We can also explore the law of sines and the law of cosines, which allow us to solve triangles using ratios of side lengths or angles. These laws have significant implications in navigation, astronomy, and construction.
If you're ready for a challenge, we could also tackle some advanced trigonometry topics like parametric equations, polar coordinates, and triple integrals.
Or, if you'd prefer a break from math, we could venture into other areas of science or explore real-world applications of trigonometry in fields like medicine, computer graphics, or music!
What sounds appealing to you?

Additional Information

$ python --version
Python 3.10.9

pip freeze | grep autogen
pyautogen==0.2.26

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.4 LTS
Release:    22.04
Codename:   jammy

[ekzhu]

Thanks for the issue. I think it makes sense to customize the role value for summary method. cc @qingyun-wu

You are welcome to create a PR for this.

[ekzhu]

2527 addressed this