microsoft / semantic-kernel

Integrate cutting-edge LLM technology quickly and easily into your apps
https://aka.ms/semantic-kernel
MIT License
21.98k stars 3.28k forks source link

.Net: RC1 support for parameter name attribute #4006

Open dluc opened 11 months ago

dluc commented 11 months ago

Given a native function such as:

[KernelFunction, Description("...")]
public async Task<string> MethodName(
    [Description("...")] string input,
    [SKName("testVar"), Description("...")] string? fooBar = null)
{ ... }

where function arguments are given a custom name: looks like RC1 is missing a port of SKNameAttribute, blocking some upgrades from beta8 to RC1.

stephentoub commented 11 months ago

where function arguments are given a custom name

An explicit decision was made to remove that (cc: @matthewbolanos). What's the scenario where you need to give it a name that you can't just use as the parameter name?

dluc commented 11 months ago

here's an example:

var args = new KernelArguments
{
    [MemoryPlugin.FilePathParam] = DocFilename,
    [MemoryPlugin.DocumentIdParam] = "NASA001"
};
await memoryPlugin["SaveFile"].InvokeAsync(kernel, args);

When doing this over and over, it's better using constants than hard coding param names that could change over time. Plus (separate feature) there's a need to allow overriding descriptions and names to get better results from planners (fewer tokens, better plans, etc.), so there should be an infrastructure that allows to manage the mapping from C# var name to external names.

stephentoub commented 11 months ago

Plus (separate feature) there's a need to allow overriding descriptions and names to get better results from planners (fewer tokens, better plans, etc.)

[Description(...)] still works on parameters.

When doing this over and over, it's better using constants than hard coding param names that could change over time.

I don't see why that requires SKName. You just name the parameter filePath and you have a constant exposed that returns filePath.

dluc commented 11 months ago

Without a mechanism to link the constant to the var name, that's bound to lead to errors, e.g. while refactoring code, without any mechanism to find them. About planners, it's not only about description, also the var name affects LLM's ability to choose a function and pass the correct data, so for instance it's better using "limit" instead of "l", and there's a case for LLMs trained on specific foreign languages, so var names need to be translatable for planners to work with more precision. (side note: most LLMs are English centric, causing up to 8x more tokens when processing foreign data, so there are models being trained differently, e.g. that's the case with Chinese, Hindi, Arabic, etc)

stephentoub commented 11 months ago

Without a mechanism to link the constant to the var name, that's bound to lead to errors, e.g. while refactoring code, without any mechanism to find them.

I don't think that's worth needing to complicate the programming model. If someone is really worried about that, they can enforce that with an analyzer, no? It doesn't require that SK supply it: they can use whatever mechanism they want to ensure that their constant value matches their parameter name.

About planners, it's not only about description, also the var name affects LLM's ability to choose a function and pass the correct data, so for instance it's better using "limit" instead of "l"

So we can name the parameter limit rather than l. I'm still not clear why the C# parameter name can't just be the parameter name.

To me this seems like unnecessary complication for little gain. But I'll defer to @matthewbolanos.

matthewbolanos commented 10 months ago

We've gotten some additional feedback from the Bing team on the desire for overriding the parameter names. The most compelling case is so that you can use prompt engineering to improve the parameter names over time (and so you aren't locked into language specific naming conventions).

matthewbolanos commented 5 months ago

Bumping further since you often need to change the names of parameter names to snake case to improve the model's ability to call the function