LLM change doesn't work

[Jpickard1]

Describe the bug The user is not able to correctly change the LLM. The LLM is changed, but this change never impacts the way that the Agent's LLM is set. In particular, the /api/llm/set endpoint was broken when the Agent factory was introduced. Prior to this, the Agent brad was treated as a global, so /api/llm/set was used to change the LLM associated with the Agent. Now each time the AgentFactory is used, a new Agent is created with the default LLM, despite the appearance of it having changed.

Proposed Solution The Agent LLM must be set whenever /api/invoke/ is accessed. Perhaps, we could have a global dictionary with keys of the session id and values of the LLM / LLM name. Then, each time we hit /invoke/ we could get the correct LLM prior to using brad.invoke()

@imnuvi What do you think?

[imnuvi]

@Jpickard1 I have used flask's SimpleCache mechanism for maintaining states previously. We can use the same mechanism to track the agent LLM. This is better than implementing our own dictionary as cache, because we can switch it out for something better later if we want.

[Jpickard1]

Sounds good. Can you make that work? I don't understand the SimpleCache mechanism @imnuvi