Need some explanation regarding funation call

[Maunish-dave]

can you help me understand why you are calling the function from the front end? can we not just call the function from the backend get the data we need and return it to response? That might save an extra api call, right? Or am I missing something here? My understanding is that the answer to my question is in this part of the code, but not sure about what this code does

// if there's a function call, we call the function and get the answer, incorporate into the history, and  call the function again
  else {
    this.handleFunctionCall(functionCallSignal)
      .then((functionCallOutput) => {
        // Then we need to call generateAnswer but with function call deactivated and a new parameter:
        this.messages.push(
          functionCallOutput
        )
        this.generateAnswer(
          chatHistory=this.getChatHistory(),
          // function_call=false,
          function_call=true,
        )
      })
      .catch((error) => {
        console.log("Error in function call:", error);
        this.botTyping=false
      });
  }

[lucastononro]

Great point! Usually the function call is just a flag of function that is raised by the OpenAI GPT Api. The function call handlers in the backend, (e.g. in RAG) are reponsible for getting the right information and giving it back to the frontend. In the frontend we identify these function callings that were flagged by the backend and then activate the responsible feature accordingly.

E.g.

User types: "Find me a good restaurant" -> The query goes to the backend and then to OpenAI API -> OpenAI API answers with a function call flagging for RAG -> In the backend I have a function call handler that activates the search process -> It returns to the frontend with the restaurant found and formulates a new answer GPT returns: "I've found the restaurant XPTO, would you like me to open it?"

User types: "Yes, open it" -> The query goes to the backend and then to OpenAI API -> OpenAI API answers with a function call flagging for Opening a restaurant -> Handler returns with the info about the restaurant to be open (arguments of the function calling - pretty dummy function call handler here) -> It returns to the frontend with the restaurant to be open -> Frontend function calling handler is flagged and activated -> Opens the given restaurant in the frontend.

Does it help you to understand it a little bit better?

[Maunish-dave]

Ok, so what we are handling here is that we are first asking the user that this is the action that I will be running so do you want to run it or not and only then we will perform that action right? And even if the user is directly asking to perform an action we will respond with "performing abc action now" and then the front-end will call for that function right?

[lucastononro]

We are mostly identifying what the user wants using GPT as a decision-maker. If you query: "Find me a good restaurant and open it", the flow will chain two function callings in a row (potentially)

User input: "Find me a good restaurant and open it" -> Goes to backend RAG Function call triggered -> Goes to frontend and makes a second API Call -> Goes to backend and Open restaurant Function call triggered -> Goes to frontend and makes the restaurant Opening handling (triggers the open restaurant function in the frontend) -> Goes back to the backend and formulates an answer Answers: "Ok, I've found the restaurant XPTO and opened it, would you like me to explore the menu?"

Here, you see, we have 3 API calls (2 that trigger functions and one that answers the user)

We do not answer with: "performing action abc" We display this as a placeholder given the action trigered by the function calling ;) because once the function is defined with have a placeholder msg for each function in the frontend

 handleOpenRestaurant: function(functionCallResponse){
        this.botTypingMsg = "Opening restaurant page..."
        // restaurantId in string format to int
        let restaurantId = parseInt(functionCallResponse.data.response.response.restaurant_uuid)
        this.$refs.restaurantsContainer.selectRestaurant(parseInt(restaurantId))
        let msg = "@agent-action: You opened the restaurant page for " + restaurantId + "!";
        console.log(msg)
        return msg
      },

So we have here "botTypingMsg" -> for each handler that exhibits the action the bot is performing and we call the function "selectRestaurant" based on the ID identified in the backend we then register the action in an array of messages (for later use, if the model judges that it need to know the actions for making a future well-oriented decision)

[lucastononro]

But it is important to note that: Function Calling + Answer -> will always take at least two API calls, that is because of the way that OpenAI API is designed to do so

[Maunish-dave]

Ah, I see. Got it. Thanks for the explanation. By the way, it is a great project, I really enjoyed going through your code.