Azure LLM Exception

[rmonvfer]

TL;DR

I've found an AttributeError: 'AzureLLMService' object has no attribute '_endpoint' error when attempting to use the AzureLLMService instead of the BaseOpenAILLMService in one of the provided example projects.

The error appears to be an instance of using an uninitialised attribute. The AzureLLMService constructor calls the parent BaseOpenAILLMService constructor before initialising its own AttributeError attribute, resulting in an AttributeError when the parent constructor tries to access the non-existent _endpoint

Detailed report

Environment

I'm using an M3 Pro MacBook Pro running MacOS Sonoma 14.1 (23B2073) and Python 3.11.7 using a virtual environment.

Reproduction steps

I've been trying to run the simple-chatbot example project but using the Azure OpenAI Service by replacing the provided BaseOpenAILLMService with AzureLLMService and updating the appropriate environment variables where required.

The resulting invocation (bot.py line 108) looks like:

llm = AzureLLMService(
    api_key=os.getenv("AZURE_OAI_KEY"),
    endpoint=os.getenv("AZURE_OAI_ENDPOINT"),
    model=os.getenv("AZURE_OAI_MODEL"),
)

Granted that all other required environment variables are provided and correct (they are), running the server.py as instructed by the README.md successfully runs the FastAPI server.

The actual exception is raised when accessing the /start endpoint via web browser, which launches the Daily UI but prevents the bot from joining the room.

Expected behaviour

The example project should work just fine by replacing the BaseOpenAILLMService with the AzureLLMService.

Actual behaviour

An exception is being raised:

  File "[REDACTED]/pipecat/examples/simple-chatbot/bot.py", line 108, in main
    llm = AzureLLMService(
          ^^^^^^^^^^^^^^^^
  File "[REDACTED]/pipecat/src/pipecat/services/azure.py", line 81, in __init__
    super().__init__(api_key=api_key, model=model)
  File "[REDACTED]/pipecat/src/pipecat/services/openai.py", line 63, in __init__
    self.create_client(api_key=api_key, base_url=base_url)
  File "[REDACTED]/pipecat/src/pipecat/services/azure.py", line 90, in create_client
    azure_endpoint=self._endpoint,
                   ^^^^^^^^^^^^^^
AttributeError: 'AzureLLMService' object has no attribute '_endpoint'

Probable Cause

The current implementation for the AzureLLMService is:

class AzureLLMService(BaseOpenAILLMService):
    def __init__(
            self,
            *,
            api_key,
            endpoint,
            api_version="2023-12-01-preview",
            model
    ):
        super().__init__(api_key=api_key, model=model)
        self._endpoint = endpoint
        self._api_version = api_version
        self._model: str = model

Its parent class is the BaseOpenAILLMService (abbreviated):

class BaseOpenAILLMService(LLMService):
    def __init__(self, model: str, api_key=None, base_url=None):
        super().__init__()
        self._model: str = model
        self.create_client(api_key=api_key, base_url=base_url)

The AttributeError observed when using the AzureLLMService class can be attributed to the order of attribute initialisation and method invocation in the inheritance hierarchy.

When an instance of AzureLLMService is created, the constructor (__init__ method) of AzureLLMService is invoked. The first line of this constructor calls the constructor of its parent class, BaseOpenAILLMService, using super().__init__(api_key=api_key, model=model). This invokes the parent constructor before the _endpoint attribute is initialised in the AzureLLMService constructor.

The BaseOpenAILLMService constructor, in turn, calls the create_client method (self.create_client(api_key=api_key, base_url=base_url)). Due to the principles of inheritance and polymorphism, the create_client method of the AzureLLMService class is invoked, as it overrides the create_client method of the parent class.

However, at this point, the _endpoint attribute has not yet been initialised in the AzureLLMService constructor, as the self._endpoint = endpoint line has not been executed. Therefore, when the create_client method of AzureLLMService tries to access self._endpoint, it raises an AttributeError because the _endpoint attribute does not exist.

This sequence of events violates the expected initialisation order and results in attempting to access an attribute before it has been properly initialised, leading to the AttributeError.

Recommended solution

Given the current implementation, I suggest to simply move the super().__init__(api_key=api_key, model=model) call to the very end of the AzureLLMService constructor so that all required attributes can be defined BEFORE they are required.

The following implementation works perfectly and passes the Azure LLM Integration Tests.

class AzureLLMService(BaseOpenAILLMService):
    def __init__(
            self,
            *,
            api_key,
            endpoint,
            api_version="2023-12-01-preview",
            model
    ):

        self._endpoint = endpoint
        self._api_version = api_version
        self._model: str = model

        super().__init__(api_key=api_key, model=model)

I'll be happy to open a PR for this, just wanted to be sure first. Please let me know if you have any questions.

[aconchillo]

Great catch @rmonvfer ! Yes, please submit a PR. That's definitely the issue! Same would happen with self._api_version.

[aconchillo]

Just went ahead and created a PR. Thank you for reporting this!