.Net: Reporting a few problems (Filters, Handlebars, general design, etc.)

[zsogitbe]

• [ ] I have tried to find the way to set the prompt format of a model, for example, some models need "<|user|>" instead of "User:", but without success. Is this an important missing feature or I have not found it?
• [ ] Filters: using Handlebars together with the Filter hooks do not seem to work at all. The callbacks are not called.

  builder.Services.AddSingleton<IFunctionFilter, FunctionFilter>();
  builder.Services.AddSingleton<IPromptFilter, PromptFilter>();

• [ ] Handlebars: it is very difficult to debug problems (there are a lot of problems). We would need some hooks to be able to get info from each step before the prompt is made. The problems include token overflow, wrong syntax, several Handlebars blocks, etc. Some problems are caused by the wrong order of messages in the PlanPrompt which seems a bit chaotic.
• [ ] We would need the possibility to correct the prompt before sending it to the model just after Handlebars made the prompt from the plan. We would need a hook there.
• [ ] HandlebarsPlannerOptions: the design should be changed to have generic options for HandlebarsPlanner which do not depend on the model. For example, this is not OK:

                      ExecutionSettings = new OpenAIPromptExecutionSettings()
                      {
                          ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions
                      },

                  this should not be OpenAI dependent here!

• [ ] Let us assume that I have a math plugin with functions add, subtract, divide, etc. Handlebars creates a template like this:

  
  ### number1-number:
  {
  "type": "number",
  "description": "The number to take the sine of"
  }
  
  ### number2-number:
  {
  "type": "number",
  "description": "The second number to subtract away"
  }
  
  ### AddReturns-number:
  {
  "type": "number"
  }
  
  ### DivideReturns-number:
  {
  "type": "number"
  }

   Notice the following problems:
  - the parameters for each function number1 and number2 are added (### number1-number, ### number2-number:), but the description gets the description from one of the functions. This will confuse the ML model. It is better to remove the description or replace it with a general description about the parameter itself, for example, "a double precision number".
  - notice that for each function the return type is added which causes an endless list of definitions and filling the context and confusing the ML model. Should replace this logic with only one general definition per return type (number, text, ...).

- [ ] the parameters of the above simple functions are simple types (double), but they are defined as complex types in the GetAvailableFunctionsManual() function => {{> ComplexTypes }}. Why? The complex types section should be empty in this case, but it lists endless double value definitions.

[zsogitbe]

I am trying to test the Phi model from Microsoft with Semantic Kernel, but it generates garbage probably because it needs "<|user|>" instead of "User: ", "<|assistant|>" instead of "Assistant: " and I am not sure about the "System: " role because Phi does not seem to have one.

I have tried to find a way to replace SK roles with Phi roles, but without success. Is this possible? If, yes how, please. Thank you.

[github-actions[bot]]

This issue is stale because it has been open for 90 days with no activity.

[rosieks]

I fully agree on that:

HandlebarsPlannerOptions: the design should be changed to have generic options for HandlebarsPlanner which do not depend on the model. For example, this is not OK: ExecutionSettings = new OpenAIPromptExecutionSettings() { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions }, this should not be OpenAI dependent here!

And it's not only regarding HandlebarsPlanner but in all cases. What's the point of having abstraction for IChatCompletion if I have to pass implementation specific settings anyway.