.Net: JSON parsing issue when you use HandlebarsPlanner

[qmatteoq]

Describe the bug I have a sample project which uses Semantic Kernel which includes 3 plugins:

• A plugin with two native functions, which are used to retrieve statistical data about the United States
• A plugin with a prompt function, which is used to generate a business mail

I'm trying to use the HandlebarsPlanner to generate a plan that can satisfy the following ask:

Write a mail to share the population of the United States in 2015. Make sure to include the number of people who identify themselves as male.

This is my code:

var planner = new HandlebarsPlanner(new HandlebarsPlannerConfig()
{
    AllowLoops = true
});

string ask = "Write a mail to share the population of the United States in 2015. Make sure to include the number of people who identify themselves as male.";

var plan = await planner.CreatePlanAsync(kernel, ask);
var result = plan.Invoke(kernel, new KernelArguments());
Console.WriteLine(result);

By exploring the result, I can see that the generated plan looks correct. In the custom helpers section, in fact, I can see that it picked up my 3 functions: GetPopulation, GetMalePopulation and WriteBusinessMail.

<system~>## Instructions
Explain how to achieve the user's goal with the available helpers with a Handlebars template.

## Example
If the user wanted you to generate 10 random numbers and use them in another helper, you could answer with the following.</system~>
<user~>Please show me how to write a Handlebars template that achieves the following goal.

## Goal
I want you to generate 10 random numbers and send them to another helper.
</user~>
<assistant~>Here's the Handlebars template that achieves the goal:
```handlebars
{{!-- Step 1: Initialize the count --}}
{{set
  "count"
  10
}}
{{!-- Step 2: Loop using the count --}}
{{#each
  (range
    1
    (get
      "count"
    )
  )
}}
  {{set
    "index"
    this
  }}
  {{!-- Step 3: Create random number --}}
  {{set
    "randomNumber"
    (Example-Random
      seed=(get
        "index"
      )
    )
  }}
  {{!-- Step 4: Call example helper with random number and print the result to the screen --}}
  {{json
    (Example-Helper
      input=(get
        "randomNumber"
      )
    )
  }}
{{/each}}
```</assistant~>
<system~>Now let's try the real thing.</system~>
<user~>Please show me how to write a Handlebars template that achieves the following goal with the available helpers.

## Goal
Write a mail to share the population of the United States in 2015

## Out-of-the-box helpers
The following helpers are available to you:
- `{{#if}}{{/if}}`
- `{{#unless}}{{/unless}}`
- `{{#each}}{{/each}}`
- `{{#with}}{{/with}}`

## Loop helpers
If you need to loop through a list of values with `{{#each}}`, you can use the following helpers:
- `{{range}}` – Generates a sequence of integral numbers within a specified range, inclusive of last value.
- `{{array}}` – Generates an array of values from the given values.

IMPORTANT: `range` and `array` are the only supported data structures. Others like `hash` are not supported. Also, you cannot use any methods or properties on the built-in data structures, such as `array.push` or `range.length`.

## Math helpers
If you need to do basic operations, you can use these two helpers with numerical values:
- `{{add}}` – Adds two values together.
- `{{subtract}}` – Subtracts the second value from the first.

## Comparison helpers
If you need to compare two values, you can use the `{{equal}}` helper.
To use the math and comparison helpers, you must pass in two positional values. For example, to check if the variable `var` is equal to number `1`, you would use the following helper like so: `{{#if (equal var 1)}}{{/if}}`.

## Variable helpers
If you need to create or retrieve a variable, you can use the following helpers:
- `{{set}}` – Creates a variable with the given name and value. It does not print anything to the template, so you must use `{{json}}` to print the value.
- `{{get}}` – Retrieves the value of a variable with the given name.
- `{{json}}` – Generates a JSON string from the given value; no need to use on strings.
- `{{concat}}` – Concatenates the given values into a string.

## Custom helpers
Lastly, you also have the following Handlebars helpers that you can use:

### `{{UnitedStatesPlugin-GetPopulation}}`
Description: Get the United States population for a specific year
Inputs:
    - year: Int32 - The year (required)
Output: String

### `{{UnitedStatesPlugin-GetMalePopulation}}`
Description: Get the United States population who identifies as male for a specific year
Inputs:
    - year: Int32 - The year (required)
Output: String

### `{{MailPlugin-WriteBusinessMail}}`
Description: Write a business mail
Inputs:
Output: string

IMPORTANT: You can only use the helpers that are listed above. Do not use any other helpers that are not listed here. For example, do not use `{{log}}` or any `{{Example}}` helpers, as they are not supported.</user~>
<system~>## Tips and tricks
- Add a comment above each step to describe what the step does.
- Use the `{{set}}` and `{{get}}` helpers to save and retrieve the results of another helper so you can use it later in the template without wasting resources.
- There are no initial variables available to you. You must create them yourself using the `{{set}}` helper and then access them using `{{get}}`.
- Do not make up values. Use the helpers to generate the data you need or extract it from the goal.
- Keep data well-defined. Each variable should have a unique name. Create and assign each variable only once.
- Be extremely careful about types. For example, if you pass an array to a helper that expects a number, the template will error out.
- Avoid using loops. Try a solution without before you deploy a loop.
- There is no need to check your results in the template.
- Do not nest sub-expressions or helpers because it will cause the template to error out.
- Each step should contain only one helper call.

## Start
Now take a deep breath and accomplish the task:
1. Keep the template short and sweet. Be as efficient as possible.
2. Do not use helpers or functions that were not provided to you, and be especially careful to not assume or make up any helpers or operations that were not explicitly defined already.
3. If none of the available helpers can achieve the goal, respond with "Additional helpers may be required".
4. The first steps should always be to initialize any variables you need.
5. The template should use the {{json}} helper at least once to output the result of the final step.
6. Don't forget to use the tips and tricks otherwise the template will not work.
7. Don't close the ``` handlebars block until you're done with all the steps.</system~>

However, when I try to execute the plan by calling InvokeAsync(), I'm getting the following exception:

System.Text.Json.JsonException: 'The JSON value could not be converted to System.Threading.Tasks.Task`1[System.String]. Path: $ | LineNumber: 0 | BytePositionInLine: 66.'

To Reproduce You can use the following sample code to reproduce the problem: https://github.com/qmatteoq/SemanticKernel-Demos/blob/rc/SemanticKernel.Planner/Program.cs

Expected behavior The plan is executed successfully.

Platform

• OS: Windows
• IDE: Visual Studio 2022 (17.8.3)
• Language: C#
• Source: NuGet package version RC3

[matthewbolanos]

This appears to be an issue with how the Handlebars planner handles async methods. Given a plugin that looks like the following...

public class UnitedStatesPlugin
{
      [KernelFunction, Description("Get the United States population for a specific year")]
      public async Task<string> GetPopulation([Description("The year")] int year)
      {
          return "124,000";
      }

      [KernelFunction, Description("Get the United States population who identifies as male for a specific year")]
      public async Task<string> GetMalePopulation([Description("The year")] int year)
      {
          return "61,000";
      }
  }

An error is thrown because somewhere the returned string is trying to be converted into a Task<string>

[teresaqhoang]

This should be fixed with this PR: https://github.com/microsoft/semantic-kernel/pull/4017

[teresaqhoang]

Tagging an issue @lemillermicrosoft reported offline as another test case.

{{!-- Step 1: Set the search query --}}
{{set "query" "latest rivian news"}}

{{!-- Step 2: Use the bing-Search helper to perform the search --}}
{{set "searchResults" (bing-Search query=(get "query") count=5)}}

{{!-- Step 3: Output the search results --}}
{{json (get "searchResults")}}

Unhandled exception: System.Text.Json.JsonException: The JSON value could not be converted to System.Threading.Tasks.Task`1[System.String]. Path: $ | LineNumber: 0 | BytePositionInLine: 209.
   at System.Text.Json.ThrowHelper.ThrowJsonException_DeserializeUnableToConvertValue(Type propertyType)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsObject(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadFromSpanAsObject(ReadOnlySpan`1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable`1 actualByteCount)
   at System.Text.Json.JsonSerializer.ReadFromSpanAsObject(ReadOnlySpan`1 json, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize(String json, Type returnType, JsonSerializerOptions options)
   at Microsoft.SemanticKernel.Planning.Handlebars.HandlebarsTemplateEngineExtensions.InvokeSKFunction(Kernel kernel, KernelFunction function, KernelArguments state, CancellationToken cancellationToken)

[teresaqhoang]

Closing this as a dupe of https://github.com/microsoft/semantic-kernel/issues/4157: .Net: Handlebars fail to process correctly result of asynchronous functions (when return type is Task<...>)