search

microsoft / semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps
https://aka.ms/semantic-kernel
MIT License
21.9k stars 3.26k forks source link

Python: Bug: The 'get_chat_message_contents' returns with a tool_calls message without calling the plugin's function(s) (intermittently) #8712

Open samelhousseini opened 1 month ago

samelhousseini commented 1 month ago

Describe the bug This is an intermittent problem and hard to reproduce with exact steps.

To Reproduce Steps to reproduce the behavior: The "chat_completion.get_chat_message_contents" function call sometimes just returns with a tool_calls message but without calling the plugin's functions described in the tool_calls. The chat history indicates that the last message added is a "tool_calls" but no function result message is added afterwards.

Expected behavior Since there's a tool calls, the expected behavior is that the function call workflow proceeds as expected with a function call and a function result, but then all I get is a tool_calls message from "chat_completion.get_chat_message_contents"

result = (await self.chat_completion.get_chat_message_contents(
                chat_history=self.history,
                settings=self.execution_settings,
                kernel=self.kernel,
                arguments=KernelArguments(),
                temperature=0.2
            ))[0]

When inspecting the results above, the get_chat_message_contents is returning without calling the plugin's functions:

image

Screenshots Screenshot below showing that the call to get_chat_message_contents just returned with an empty string, because the ChatMessageContent is a function call "tool_calls".

print("Assistant > " + str(result))

And that the call to the filter did not happen.

Platform

image

Update:

The following block of code is a temp fix of this problem:

        counter = 0

        while True:
            print(f"-------------- Attempting response {counter}")
            result = (await self.chat_completion.get_chat_message_contents(
                chat_history=self.history,
                settings=self.execution_settings,
                kernel=self.kernel,
                arguments=KernelArguments(),
                temperature=0.2
            ))[0]

            if result.finish_reason == 'tool_calls':
                self.history.add_message(result)
                function_results = []

                for item in result.items:
                    content = await self.kernel.plugins[item.plugin_name].functions[item.function_name].method(**json.loads(item.arguments))
                    print("Content from Function Call - ", content)
                    function_results.append(
                        FunctionResultContent(
                            id=item.id, name=item.name, result=str(content)
                        )
                    )                    

                self.history.add_message(
                    ChatMessageContent(
                        role = AuthorRole.TOOL,
                        items = function_results
                    )
                )
            else:
                break

            counter += 1

            if counter > 25: break

        # Add the message from the agent to the chat history
        self.history.add_message(result)
eavanvalkenburg commented 1 month ago

@samelhousseini what are the settings you are using?

samelhousseini commented 1 month ago

Hi @Eduard van @.***>,

I believe you meant the execution settings? Please find below.

    self.execution_settings = AzureChatPromptExecutionSettings(tool_choice="auto")
    self.execution_settings.function_choice_behavior = FunctionChoiceBehavior.Auto()

Many thanks

Samer

From: Eduard van Valkenburg @.> Sent: Monday, September 16, 2024 6:11 PM To: microsoft/semantic-kernel @.> Cc: Mention @.>; Author @.> Subject: Re: [microsoft/semantic-kernel] Python: Bug: The 'get_chat_message_contents' returns with a tool_calls message without calling the plugin's function(s) (intermittently) (Issue #8712)

@samelhousseinihttps://github.com/samelhousseini what are the settings you are using?

— Reply to this email directly, view it on GitHubhttps://github.com/microsoft/semantic-kernel/issues/8712#issuecomment-2353041719 or unsubscribehttps://github.com/notifications/unsubscribe-auth/AZ2RAUXNZ3WJHMNQB7ODC3TZW3RJXBFKMF2HI4TJMJ2XIZLTS2BKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLAVFOZQWY5LFVI2TEMBVGAYTIMJVHCSG4YLNMWUWQYLTL5WGCYTFNSBKK5TBNR2WLKRVGI4DSNJTHEYTCOFENZQW2ZNJNBQXGX3MMFRGK3ECUV3GC3DVMWVDOMBWGM4DMMZYGU4KI3TBNVS2S2DBONPWYYLCMVWKY43VMJVGKY3UL52HS4DFVREXG43VMVBW63LNMVXHJJTUN5YGSY3TSWBKI5DZOBS2U4TFOBXXG2LUN5ZHTJLWMFWHKZNJGYYDOMRYHEYTQNMCUR2HS4DFUVUXG43VMWSXMYLMOVS2UMRVGI2DOOJSGE2DDAVEOR4XAZNFNRQWEZLMUV3GC3DVMWVDKMRQGUYDCNBRGU4IFJDUPFYGLJLMMFRGK3FFOZQWY5LFVI2TEOBZGUZTSMJRHCBKI5DZOBS2K3DBMJSWZJLWMFWHKZNKG4YDMMZYGYZTQNJYU52HE2LHM5SXFJTDOJSWC5DF. You are receiving this email because you were mentioned.

Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.