If I am a user and I create a workspace and send an @agent invocation the logic for provider/model selection for agent execution should be:
use the explicitly set agentProvider and agentModel combination. No verification is done to confirm the pairing is valid, just that it exists. The API will return any errors anyway.
Fallback to the workspaces chatProvider and chatModel overrides, this at least keeps answers within the same model preference the user-defined for a workspace.
Lastly, attempt to use the LLM_PROVIDER env and its ENV default model. If no ENV preference can be found - assume a static model string that is generally available for the provider. If that cannot be determined - return null and probably exit since most LLM providers do not accept null for model.
This covers every case of agent invocation without the user needing to specify anything to unlock agents since our coverage is mostly complete now anyway. The user should not have to set an agent provider to use agents, just like how chatting works. You can define a model for a per-workspace preference, but it is not required.
TODO: we may have fallback procedure not failover to chatprovider/model since that can be confusing, but we can revist
How are you running AnythingLLM?
All versions
What happened?
When we implemented https://github.com/Mintplex-Labs/anything-llm/issues/2353 this was an incomplete implementation of the fallback logic for agent selection.
If I am a user and I create a workspace and send an
@agentinvocation the logic for provider/model selection for agent execution should be:use the explicitly set
agentProviderandagentModelcombination. No verification is done to confirm the pairing is valid, just that it exists. The API will return any errors anyway.Fallback to the workspaces
chatProviderandchatModeloverrides, this at least keeps answers within the same model preference the user-defined for a workspace.Lastly, attempt to use the
LLM_PROVIDERenv and its ENV default model. If no ENV preference can be found - assume a static model string that is generally available for the provider. If that cannot be determined - return null and probably exit since most LLM providers do not acceptnullfor model.This covers every case of agent invocation without the user needing to specify anything to unlock agents since our coverage is mostly complete now anyway. The user should not have to set an agent provider to use agents, just like how chatting works. You can define a model for a per-workspace preference, but it is not required.
TODO: we may have fallback procedure not failover to chatprovider/model since that can be confusing, but we can revist
Are there known steps to reproduce?
No response