Open dluc opened 11 months ago
where function arguments are given a custom name
An explicit decision was made to remove that (cc: @matthewbolanos). What's the scenario where you need to give it a name that you can't just use as the parameter name?
here's an example:
var args = new KernelArguments
{
[MemoryPlugin.FilePathParam] = DocFilename,
[MemoryPlugin.DocumentIdParam] = "NASA001"
};
await memoryPlugin["SaveFile"].InvokeAsync(kernel, args);
When doing this over and over, it's better using constants than hard coding param names that could change over time. Plus (separate feature) there's a need to allow overriding descriptions and names to get better results from planners (fewer tokens, better plans, etc.), so there should be an infrastructure that allows to manage the mapping from C# var name to external names.
Plus (separate feature) there's a need to allow overriding descriptions and names to get better results from planners (fewer tokens, better plans, etc.)
[Description(...)] still works on parameters.
When doing this over and over, it's better using constants than hard coding param names that could change over time.
I don't see why that requires SKName. You just name the parameter filePath
and you have a constant exposed that returns filePath
.
Without a mechanism to link the constant to the var name, that's bound to lead to errors, e.g. while refactoring code, without any mechanism to find them. About planners, it's not only about description, also the var name affects LLM's ability to choose a function and pass the correct data, so for instance it's better using "limit" instead of "l", and there's a case for LLMs trained on specific foreign languages, so var names need to be translatable for planners to work with more precision. (side note: most LLMs are English centric, causing up to 8x more tokens when processing foreign data, so there are models being trained differently, e.g. that's the case with Chinese, Hindi, Arabic, etc)
Without a mechanism to link the constant to the var name, that's bound to lead to errors, e.g. while refactoring code, without any mechanism to find them.
I don't think that's worth needing to complicate the programming model. If someone is really worried about that, they can enforce that with an analyzer, no? It doesn't require that SK supply it: they can use whatever mechanism they want to ensure that their constant value matches their parameter name.
About planners, it's not only about description, also the var name affects LLM's ability to choose a function and pass the correct data, so for instance it's better using "limit" instead of "l"
So we can name the parameter limit
rather than l
. I'm still not clear why the C# parameter name can't just be the parameter name.
To me this seems like unnecessary complication for little gain. But I'll defer to @matthewbolanos.
We've gotten some additional feedback from the Bing team on the desire for overriding the parameter names. The most compelling case is so that you can use prompt engineering to improve the parameter names over time (and so you aren't locked into language specific naming conventions).
Bumping further since you often need to change the names of parameter names to snake case to improve the model's ability to call the function
Given a native function such as:
where function arguments are given a custom name: looks like RC1 is missing a port of
SKNameAttribute
, blocking some upgrades from beta8 to RC1.