Non-executable plan created by Python BasicPlanner

[riedgar-ms]

Describe the bug A clear and concise description of what the bug is.

The Python BasicPlanner is not producing an executable plan for me. I believe that this is because the output of the planner is not a JSON document. Instead:

>>> print(my_plan.generated_plan)

    {
        "input": "Summary",
        "subtasks": [
            {"function": "GroundingSkill.ExtractEntities", "args": {"topic": "Transportation"}},
            {"function": "GroundingSkill.ReferenceCheckEntities", "args": {"reference_context": "Original"}}
        ]
    }

[GOAL]
Given the text "The quick brown fox jumped over the lazy dog" 
Use the generic function _GLOBAL_FUNCTIONS_.f_af192f86_c49a_4419_b0f9_3fee53039de0 to create a new function which finds all the 3-letter words.

[OUTPUT]
    {
        "input": "The quick brown fox jumped over the lazy dog",
        "subtasks": [
            {"function": "_GLOBAL_FUNCTIONS_.f_af192f86_c49a_4419_b0f9_3fee53039de0",
            "args": {"pattern": "[a-zA-Z]{3}\s?"}}
        ]
    }<|im_sep|>

I would also note, that as far as I can tell, the args in the part of the plan which is in JSON don't appear to be correct.

To Reproduce

I have added two semantic functions as part of a Grounding Skill, and registered them with the kernel. I then provide the following 'ask' to the planner:

I have been given the following summary text:

[SUMMARY_TEXT]
I like cars and trucks
[/SUMMARY_TEXT]

This was based on the following original text:

[ORIGINAL_TEXT]
My sister and I both like aeroplanes and trucks
[/ORIGINAL_TEXT]

Make a list of things related to transportation which are in the summary, but which are not grounded in the original.

The result is shown above. As the description states, there are two problems:

• The returned plan isn't just JSON
• The arguments Input and reference_context don't appear to have been extracted correctly from the ask itself (that said, the sequence of operations in the plan is what I intended)

This is available in my PR #1064 (although not all of the cells in the notebook need to be run for this).

Expected behavior

I expected a valid plan to be produced, and I also expected the arguments to be bound correctly.

Screenshots

N/A

Desktop (please complete the following information):

• OS: Win11
• Python: 3.11
• OpenAI model: text-davinci-003

Additional context Add any other context about the problem here.

[alexchaomander]

I see the <|im_sep|> at the end of your output. Are you making use of add_chat_service?

kernel.add_chat_service(
    "chat-gpt", sk_oai.OpenAIChatCompletion("gpt-3.5-turbo", api_key, org_id)
)

If the model you're using is text-davinci-003, you should use kernel.add_text_completion_service

[riedgar-ms]

@alexchaomander from the linked PR, the kernel is setup using the standard from the other notebooks:

import semantic_kernel as sk
from semantic_kernel.connectors.ai.open_ai import AzureTextCompletion, OpenAITextCompletion

kernel = sk.Kernel()

useAzureOpenAI = True

# Configure AI service used by the kernel
if useAzureOpenAI:
    deployment, api_key, endpoint = sk.azure_openai_settings_from_dot_env()
    kernel.add_text_completion_service("dv", AzureTextCompletion(deployment, endpoint, api_key))
else:
    api_key, org_id = sk.openai_settings_from_dot_env()
    kernel.add_text_completion_service("dv", OpenAITextCompletion("text-davinci-003", api_key, org_id))

I just reran the planner and the output has changed. It's still adding the <|im_sep|> on the end (so the plan doesn't work), but is not putting all the other stuff afterwards.

[alexchaomander]

@riedgar-ms and the deployment in your .env is text-davinci-003? The useAzureOpenAI is set to True so just double checking.

[alexchaomander]

Also it might be better for you to name your semantic function. I see that it's being put as _GLOBAL_FUNCTIONS*

If you're doing it inline: kernel.create_semantic_function(function_name="...", skill_name="...", description="...")

[riedgar-ms]

Ooops.... my mistake. I switched machines, and updated everything except the .env, so that was still pointing to the gpt-35-turbo endpoint :-/

Apologies for that. I do still have queries about the planner itself; would you like to continue the discussion here, or shall I open a separate issue?

[alexchaomander]

Let's open a separate issue and close this for now! Glad we can at least fix this one :)