Info retrieval and LLM function calling with parameter

[eric-gardyn]

I successfully added a function calling to my LLM (using mongo-chatbot-server) On the call to /conversations (and /conversations/:id/messages), I pass a user ID parameter. I store it as customData in the conversations collection. When the user's query is about a personal request, I am able to have the LLM invoke my function. However, I would like to know the context (user ID) so I can make a query from another MongoDB collection (or a SQL DB). Do you have a best practice approach?

[eric-gardyn]

I found a "hacky" way by getting the parameter from the 'request' object which is passed to the tool 'call' method, but it feels a little weird, no?

[mongodben]

your "hacky" way of doing this is the way that i designed it to work 😄

do you have any thoughts on a way that'd feel better to you?

my though with this implemention, passing the request, is in case the request contains credentials like an access token that you don't want to persist to custom user data.

---

an alternative that's not currently possible would be to parse any request credentials in middleware and include anything that you want for protected actions in the Response.locals, and then use that in the tool call. With that being said, the Response obj isn't currently accessible from the tool call function, so we'd need to add that logic.

[eric-gardyn]

I was thinking I could get the CustomData (since it is already parsing the request and augmenting it with my own data)

[mongodben]

I was thinking I could get the CustomData (since it is already parsing the request and augmenting it with my own data)

yeah that could work if whatever you're looking to include in the request you can also safely store in your DB.

[eric-gardyn]

my goal is to be able to retrieve user information that can be fed to the LLM