Some problems occur when calling the plugin

[lingxiaaisuixin]

I have a plugin that queries the data, and the plugin has two parameter. e.g. df,des = query_data(query,describe) Question 1: If you enter a question, it may be split into multiple sub-queries, and the plug-in querydata will be called several times. The plug-in needs to pass two parameters, one of which is describe, and needs to receive the result of the last call to the plug-in.The parameters are often not passed into the plug-in accurately when the plug-in is received.It might pass the returned df to the describe parameter.Sometimes it is not possible to return all the results.As follows,, only one df is returned. df,= query_data(query,describe) Question 2: In the process of calling the plug-in, my query result already contains the data needed for the query, but when calling the plug-in, some additional code snippets will be added. I hope that when calling the plug-in, it will judge whether to add additional code snippets based on the return result of my plug-in. I hope you can give me some advice on the problems I have encountered.

[ShilinHe]

Hi, for question 1, it depends on the LLM's capability to generate the code, including arguments and return values. There might be some reasons: a) the LLM is not powerful enough. b) the plugin definition is not clear, as the LLM relies on the YAML definition to generate the argument/return values c) the scenario requires a lot of domain knowledge, may need to add a few examples to guide the model. Specifically in your case, code interpreter examples should be good enough.

For question 2, not sure if I understand the question correctly. Since you want TaskWeaver to add additional code in certain cases (e.g., query result is xx), but TaskWeaver never have that specific knowledge, one possible way is to add some examples that fit your scenario.

[lingxiaaisuixin]

Thank you. The LLM I used is qwen-plus. I may not have given a clear description of question 2. In short, sometimes the LLM, without confirming whether the return result of calling the plug-in has met the user's requirements, it judged to have incomplete output of the plug-in, generating additional code snippets that I do not need.

[ShilinHe]

I see, is it because of that there is a gap between the plugin functionality description and user's query intention? I would suggest adding some examples to TaskWeaver to let it know the current plugin call is good enough and no additional code is needed.

[lingxiaaisuixin]

thanks, The whole process looks like the following: user intention->LLM A->task->plguin(LLM B->NL2SQL->data)->whether it satisfies?
It is difficult to ensure the information integrity of the entire inference link.I may have a long way to go