Bug: Handlebars planner fails to create plan

[Barshan-Mandal]

var plan = await planner.CreatePlanAsync(kernel, "get the current date and time"); throws the error below:

System.Text.Json.JsonException
  HResult=0x80131500
  Message=The JSON value could not be converted to System.String. Path: $ | LineNumber: 0 | BytePositionInLine: 1.
  Source=System.Text.Json
  StackTrace:
   at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, Utf8JsonReader& reader, Exception ex)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](JsonElement element, JsonSerializerOptions options)
   at Microsoft.SemanticKernel.Planning.Handlebars.KernelParameterMetadataExtensions.ParseJsonSchema(KernelParameterMetadata parameter)
   at Microsoft.SemanticKernel.Planning.Handlebars.HandlebarsPlanner.SetComplexTypeDefinition(KernelParameterMetadata parameter, HashSet`1 complexParameterTypes, Dictionary`2 complexParameterSchemas)
   at Microsoft.SemanticKernel.Planning.Handlebars.HandlebarsPlanner.GetAvailableFunctionsManual(IEnumerable`1 availableFunctions, HashSet`1& complexParameterTypes, Dictionary`2& complexParameterSchemas)
   at Microsoft.SemanticKernel.Planning.Handlebars.HandlebarsPlanner.<CreatePlanCoreAsync>d__5.MoveNext()
   at Microsoft.SemanticKernel.Planning.PlannerInstrumentation.<CreatePlanAsync>d__4`2.MoveNext()
   at phi3vision_aiconsole.Program.<Phi3MemoryRAG>d__1.MoveNext() in O:\Windows For Programming\Projects\Visual Studio\Console\Ai\phi3vision_aiconsole\Program.cs:line 87
   at phi3vision_aiconsole.Program.<Main>d__0.MoveNext() in O:\Windows For Programming\Projects\Visual Studio\Console\Ai\phi3vision_aiconsole\Program.cs:line 36

  This exception was originally thrown at this call stack:
    [External Code]

Inner Exception 1:
InvalidOperationException: Cannot get the value of a token type 'StartArray' as a string.

[markwallace-microsoft]

Hi @Barshan-Mandal, can you provide a basic sample showing how we can reproduce this issue? Thanks.

[Barshan-Mandal]

Hi @Barshan-Mandal, can you provide a basic sample showing how we can reproduce this issue? Thanks.

` var modelPath = @"F:\HuggingFace Models\Phi-3-Vision\cpu-int4-rtn-block-32-acc-level-4"; string modelId = "microsoft/Phi-3-vision-128k-instruct"; string bge = @"F:\OnnxModels\bge-micro-v2\model.onnx"; string vocab = @"F:\OnnxModels\bge-micro-v2\vocab.txt";

// Load the model and services var builder = Kernel.CreateBuilder(); builder.AddOnnxRuntimeGenAIChatCompletion(modelId, modelPath); // builder.AddLocalTextEmbeddingGeneration(); builder.AddBertOnnxTextEmbeddingGeneration(bge, vocab);

// Build Kernel var kernel = builder.Build();

// Create services such as chatCompletionService and embeddingGeneration var chatCompletionService = kernel.GetRequiredService(); var embeddingGenerator = kernel.GetRequiredService();

// Setup a memory store and create a memory out of it var memoryStore = new VolatileMemoryStore(); var memory = new SemanticTextMemory(memoryStore, embeddingGenerator); var planner = new HandlebarsPlanner(plannerOptions); var plan = await planner.CreatePlanAsync(kernel, "get the current date and time");`

[markwallace-microsoft]

Hi @Barshan-Mandal we haven't tested the Handlebars Planner with the ONNX AI connector. We have focussed on OpenAI and Azure OpenAI.

But we have decided to stop work on the Handlebars and Stepwise planners in favour of using code interpreter and function calling. The following Blog post explains our strategy: https://devblogs.microsoft.com/semantic-kernel/the-future-of-planners-in-semantic-kernel/

So our advise is to investigate the alternatives proposed in the Blog post. We're also happy to get any feedback you have on ONNX as we plan to graduate the connector later this year.