Wizard 7B does not follow prompt correctly

[yhyu13]

Prompt:

export const analyzeTaskPrompt = new PromptTemplate({
  template: `You have the following higher level objective "{goal}". You currently are focusing on the following task: "{task}". Based on this information, evaluate what the best action to take is strictly from the list of actions: {actions}. You should use 'search' for searching the web about current events where "arg" is a simple clear search query based on the task only. Use "reason" for all other actions. Return the response as an object of the form {{ "action": "string", "arg": "string" }} that can be used in JSON.parse() and NOTHING ELSE.`,
  inputVariables: ["goal", "actions", "task"],
});

Actual Response:

'Understood. Based on the current task, the best action to take is to search the web about current events related to "Alexander Graham Bell". Therefore, the action to take is "search" and the argument is "arg". The response will be of the form { "action": "search", "arg": "arg" }.'

Expected response

'{ "action": "search", "arg": "Alexander Graham Bell" }'

It seems Wizard LM 7B is not able to follow the exact prompted instructions. Is anyone else have the same issue?

[idosal]

How did you reach the search flow? It's disabled on https://agentllm.vercel.app/

[yhyu13]

Nah, I mod ts code to enable it. So, the search action is not expected to work?

[idosal]

Not at the moment. I will develop a local scraping capability at some point and add it.