Sending first message by agent

[agokrani]

Hi,

Is there any way for the agent to send the first message. I could customize the websocket connection to do this. However, I don't want to store the initial message that triggered the bot to be stored. Is there any other possibility of doing it with websocket connection.

[ajndkr]

hi @agokrani, apologies for delayed response. can you maybe explain your use-case here?

I don't think this idea is viable with the REST approach, maybe you have differing opinion?

Using websockets seem like a good idea but again appears to be more task-specific than a generic requirement.

[agokrani]

Hi @ajndkr,

Thank you for reaching out. I agree this is more of a task specific requirement than generic one. However, I think it will be a common request as we move towards more advanced agents, and maybe it is worth discussing as an open discussion.

To elaborate on my question, let's consider the example of SalesGPT from langchain:

https://python.langchain.com/docs/use_cases/agents/sales_agent_with_context

Imagine this as a chat agent. Now, when the user clicks a certain button, the agent should send an introduction message. I only want agents' responses to be stored and not the message sent for activating the agent. Similarly, in the case of follow-up. It should only store the agents response because that's just the first thing the user sees in the chat history.

Does this clarify my use case?

[ajndkr]

@agokrani apologies for the late response. If this issue is still relevant for you, we can discuss more. Would you like to contribute towards this? Maybe by now you've already thought about some ideas. we can brainstorm here.

[agokrani]

Hi @ajndkr,

Yes, this is still relevant, and I would definitely like to contribute towards it. I still didn't have any good approach for doing this. For example, if I use websocket than the problem is that each time a new connection is established, the LLM will send the first message, which may or may not be desired. Secondly, the langchain memory stores both input and output when LLM stops responding. One way I thought of doing this is creating an extra post request only when the first-time agent gets initialized with a custom memory object that only stores the output of LLM. I don't think this solution is going to be directly integrated to the lanarky, but it will be a good example for others to get started. What do you think about this approach? I would also like to hear your thoughts on this.

[ajndkr]

For example, if I use websocket than the problem is that each time a new connection is established, the LLM will send the first message, which may or may not be desired. Secondly, the langchain memory stores both input and output when LLM stops responding.

I'm afraid I need more clarity here. Manipulating the langchain memory is easy. At the end of each step, we can trigger a logic which updates the memory however we want. What i don't understand is your requirement about the first message. what do you mean by "may or may not be desired"? is sending the first message meant to be optional?

Maybe you can provide a sample conversation and help me understand what needs to be done? For example:

( User clicks button)
AI: hi! i'm assistant here to help you!
Human: hi! can you...
...

Something like this?

[agokrani]

Yes, the user clicks the chat button, and the LLM responds: AI: I am the sales agent writing you from XYZ company, we are having ABC sales on our products.... Human: Sounds good. Can you provide me with a catalog of your products on sale....

Yes, I want this first message to be optional that I can configure based on some condition when the LLM sends the first message or when it shouldn't

[ajndkr]

i see! okay. so if the first message is not supposed to be part of the langchain memory. then we can implement a very simple REST solution:

Create 2 API endpoints:

• POST /wake: wake up message by LLM
• POST /chat: chat with LLM

On the client side, now you can configure how you want the chatbot interaction to begin. If the bot should send the first message, hit the /wake endpoint first. Otherwise hit the /chat endpoint.

For ignoring the first message if the bot sends a message first, i think it's much easier to implement on the client side rather than the server side.

Does this work for you? I think we can avoid websockets if it's not a must-have for your requirement.

[agokrani]

Hi @ajndkr,

Sorry for the delay. I had limited access to the internet.

I think your solution makes sense. I can give it a try in the next week or so. I just have one query, when you say, "it's much easier to configure on the client side rather than server side, what do you mean by that?"

If I understand you correctly in the /wake endpoint, I can basically not use the chat memory in the chain, but store the AI message using more lower level APIs of memory. Did I miss something?

[ajndkr]

I just have one query, when you say, "it's much easier to configure on the client side rather than server side, what do you mean by that?"

I meant it's easier for the client side to make 2 API calls rather than building a complex logic for the single /chat endpoint

If I understand you correctly in the /wake endpoint, I can basically not use the chat memory in the chain, but store the AI message using more lower level APIs of memory. Did I miss something?

that's right! i think you can even create a separate simple LLMChain to return a wake up message. the human prompt can be pre-defined like "user says hi".

[ajndkr]

I am moving this to the discussions page as it's more appropriate.