microsoft / semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps
https://aka.ms/semantic-kernel
MIT License
21.73k stars 3.22k forks source link

Python: Kernel functions cannot handle list parameters. #6367

Closed q-stefanmuscalu closed 5 months ago

q-stefanmuscalu commented 5 months ago

Describe the bug I am writing a native Python plugin. Stuff breaks if I use list[str] as parameters for my kernel_function.

To Reproduce Steps to reproduce the behavior: Define this kernel function:

class AllergyPlugin:
    @kernel_function(
        name="FindAllergens",
        description="Filters a list of ingredients to find allergens."
    )
    def get_configuration(
        self,
        ingredients: Annotated[list[str], "The list of ingredients."]
    ) -> Annotated[list[str], "The list or allergens."]:
        return [ingredient for ingredient in ingredients if ingredient in ["peanuts"]]

My app starts up fine. Then when I ask "What of the following ingredients are allergens? milk, soy, salt, peanuts, strawberries, and eggs" I get the following error:

  File "/gen-ai/.venv/lib/python3.11/site-packages/semantic_kernel/functions/kernel_function_from_prompt.py", line 168, in _invoke_internal
    raise FunctionExecutionException(f"Error occurred while invoking function {self.name}: {exc}") from exc
semantic_kernel.exceptions.function_exceptions.FunctionExecutionException: Error occurred while invoking function Chat: ("<class 'semantic_kernel.connectors.ai.open_ai.services.azure_chat_completion.AzureChatCompletion'> service failed to complete the prompt", BadRequestError('Error code: 400 - {\'error\': {\'message\': "Invalid schema for function \'allergy-FindAllergens\': In context=(\'properties\', \'ingredients\'), array schema missing items", \'type\': \'invalid_request_error\', \'param\': None, \'code\': None}}'))

I have a workaround where I tell the agent to give me a string with a separator:

class AllergyPlugin:
    @kernel_function(
        name="FindAllergens",
        description="Filters a list of ingredients to find allergens."
    )
    def get_configuration(
        self,
        ingredients: Annotated[str, "The list of ingredients separated by commas. Example 'milk, soy, eggs'"]
    ) -> Annotated[list[str], "The list or allergens."]:
        allergens = ["peanuts"]
        return [ingredient for ingredient in ingredients.split(", ") if ingredient in allergens]

Then the agent correctly replies with:

Among the ingredients listed, peanuts are identified as allergens.

Expected behavior I expect kernel functions to handle list parameters.

Screenshots NA

Platform

Additional context Add any other context about the problem here.

q-stefanmuscalu commented 5 months ago

The problem is in semantic_kernel.connectors.ai.open_ai.services.utils.kernel_function_metadata_to_openai_tool_format

parse_schema needs another case for array, something like this

        elif schema_data.get("type") == "array":
            result = {
                "type": schema_data.get("type", "array"),
                "description": schema_data.get("description", ""),
                "items": { "type": "string" }
            }
alliscode commented 5 months ago

@q-stefanmuscalu thank you for filing this issue, we'll take a look at it and get it resolved.

q-stefanmuscalu commented 5 months ago

Hey, thank you for opening the pull request! I discovered that list return types don't work either. If my tool returns a list, only the first element of the list makes it to the agent. It looks like a separate issue to me.

moonbox3 commented 5 months ago

Hi @q-stefanmuscalu that issue is also fixed in #6370.