microsoft / semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps
https://aka.ms/semantic-kernel
MIT License
21.6k stars 3.2k forks source link

Python: Bug: retry_machanism is not used #9015

Open CBumann opened 2 weeks ago

CBumann commented 2 weeks ago

Describe the bug The Kernel class has a property retry_mechnism. image

This is intended to be set to a Implementation of RetryMechanismBase from the semantic_kernel.reliability extension to handle the retry logic. However this property seems to never be used, leaving us unable to handle retries in the intended way. This was tested both with the kernel.invoke functions as well as with get_chat_message_content.

To Reproduce

  1. Create Implementation of RetryMechanismBase
  2. Initialize in Kernel
  3. Run Chat Completion or function invoke Code Sample to reproduce:
    
    from semantic_kernel import Kernel
    from semantic_kernel.reliability.retry_mechanism_base import RetryMechanismBase
    from typing import Awaitable, Callable, TypeVar
    from semantic_kernel.connectors.ai.open_ai import (
    AzureChatCompletion,
    )
    from semantic_kernel.contents.chat_history import ChatHistory
    from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings
    import pytest

T = TypeVar("T")

azure_openai_endpoint = "your-endpoint" azure_openai_key = "your-key" azure_openai_model = "gpt-4o"

class FailingRetryMechanism(RetryMechanismBase): async def execute_with_retry(self, action: Callable[[], Awaitable[T]]) -> T: raise NotImplementedError

kernel = Kernel(retry_mechanism=FailingRetryMechanism()) kernel.add_service( AzureChatCompletion( deployment_name=azure_openai_model, endpoint=azure_openai_endpoint, service_id=azure_openai_model, api_key=azure_openai_key, ), )

chat_completion_service = kernel.get_service(service_id=azure_openai_model)

chat_history = ChatHistory() chat_history.add_user_message("Hello, how are you?")

with pytest.raises(NotImplementedError): response = await chat_completion_service.get_chat_message_content( chat_history=chat_history, settings = PromptExecutionSettings(), )


The test above fails, because FailingRetryMechanism is never used.

**Expected behavior**
The provided retry mechanism should be used..

**Platform**
 - OS:  Windows, Mac
 - IDE: VS Code
 - Language: Python
 - Source:package version 1.9.0
moonbox3 commented 1 week ago

@TaoChenOSU could you have a look at this, please?