Inbuilt LLM and Memory Configuration for Agents

Prerequisites

[X] I checked the documentation and made sure this feature does not already exist
[X] I checked the existing issues to make sure this feature has not already been requested

Feature

I would like to suggest adding inbuilt LLM (Large Language Model) and memory configuration options for our agents. While we currently have storage on agents, incorporating LLM and memory configuration can significantly enhance the agents' capabilities by providing reasoning and thought power, along with personalization through memory.

This feature would allow agents to become smarter and more personalized for users by enabling them to remember and use past interactions. Here is an example of how this could be implemented:

For LLM configuration:

my_llm = Ollama(
    model = "llama2",
    base_url = "http://localhost:11434"
)

alice = Agent(name="alice", seed="alice recovery phrase", llm=my_llm)

For memory configuration:

result = memory.add("I am working on improving my tennis skills. Suggest some online courses.", user_id="alice", metadata={"category": "hobbies"})
print(result)

# Search memories
related_memories = memory.search(query="What are Alice's hobbies?", user_id="alice")
print(related_memories)

Additional Information (Optional)

Implementing this feature can make the AI agents more personalized for users, enhancing their interaction experience and making the agents more useful in various applications. The LLM and memory configurations should be optional, allowing users to choose based on their specific needs.

Thanks for the suggestion! As I see it this feature would behave similar to a RAG agent in which the response or behaviour of the agent is augmented by its interaction history rather than plainly from a contextual phrase. What kind of applications do you see here?

I'm wondering if this would benefit agents on a general level or only specific use-cases since not every agent is connected to DeltaV. I would have to look at the statistics again but I'm currently assuming (also for privacy and security reasons) that users would prefer to have this as an opt-in solution and since agents are meant to be very atomic most of the times it wouldn't make sense to add overhead by putting this feature inside the main uAgents framework.

This could potentially land in the contribution module or be an extra though.

Giving others the chance to weigh in here but my position is currently against that proposal.

Thanks for the suggestion! As I see it this feature would behave similar to a RAG agent in which the response or behaviour of the agent is augmented by its interaction history rather than plainly from a contextual phrase. What kind of applications do you see here?

I'm wondering if this would benefit agents on a general level or only specific use-cases since not every agent is connected to DeltaV. I would have to look at the statistics again but I'm currently assuming (also for privacy and security reasons) that users would prefer to have this as an opt-in solution and since agents are meant to be very atomic most of the times it wouldn't make sense to add overhead by putting this feature inside the main uAgents framework.

This could potentially land in the contribution module or be an extra though.

Giving others the chance to weigh in here but my position is currently against that proposal.

@Archento , Thank you for your feedback on my feature suggestion!

I appreciate your points regarding privacy, security, and the atomic nature of most agents. However, I believe that integrating inbuilt LLM and memory configuration options could provide significant benefits, particularly in making agents more intelligent and personalized for users.

Here are some points that I think could address your concerns and highlight the potential advantages:

Opt-in Feature: I completely agree that this feature should be optional. Users could enable LLM and memory configuration based on their specific needs. This would ensure that only those who want enhanced personalization and reasoning capabilities would use it, addressing privacy and security concerns.

Enhanced Capabilities: By incorporating LLMs like "llama2" and memory configurations, agents can provide more accurate and context-aware responses. This can be particularly beneficial for applications where understanding user history and preferences can lead to better interactions, such as customer support, personal assistants, and educational tools.

Modular Implementation: To avoid adding overhead to the main uAgents framework, we could implement this as a separate module or a plugin. This way, the core framework remains lightweight, and only users who need this feature would add it to their agents.

Use Cases: While not all agents might benefit from this, there are several use cases where it could be highly advantageous. For instance, in customer support, an agent that remembers previous interactions can provide more relevant solutions. In educational tools, an agent that understands a user's progress and preferences can offer personalized learning paths.

Overall, I believe that adding inbuilt LLM and memory configuration options as an optional, modular feature can greatly enhance the capabilities of our agents. It offers users the flexibility to choose advanced functionalities while maintaining the simplicity and efficiency of the main framework.

I think it comes down to the definition of 'intelligence'. An agent is a piece of software that either wraps or attaches to your existing business logic so the agents main task (in almost every case) is to either:

provide a layer of abstraction for the communication between multiple entities, and/or
encapsulate trigger based logic where an input A will generate an output B.

In most cases the output of an agent will be deterministic meaning that you expect your agent to behave in a foreseeable fashion every time you interact with it. One simple example is an agent that needs to turn on the light when you tell it to. This should under no circumstances be influenced by preference - if you need the thing to happen when you want it to it definitely should happen.

This brings me to your example where an agent is tasked to do something but the output will be heavily if not fully influenced by a prediction or assumption based on your previous behaviour. This also implies that the interaction is multifaceted and consists of a complex graph, e.g. by using a dialogue. Or that the outcome relies on an interaction history. Sure, from a user perspective it would sometimes be nice if the agent would turn on the lights automatically when you come into the room - but even that needs a trigger of sorts (proximity sensor, bluetooth connection, etc.). In contrast if you look at the main driving force in this world which is the industry sector you can see that reliability and deterministic behaviour has prevailed. Also in this scenario the agent will shift from its role of an intermediary to the main interaction point and the more complex the scenario the more logic you need to implement in the agent anyway.

For me the solution is very simple: What you propose is a totally viable Integration. Something that every developer can simply add to their agent by copy&paste from the integrations folder (if he/she needs it) and not something that automatically comes with the agent.

So please feel free to open a PR with this addition as I'm sure it will help out some people and thank you for the suggestion :)

fetchai / uAgents