Add session ID to llm request metadata

[nsbradford]

Get sessionId from the frontend in index.tsx and pass it through sendLLMRequest in api.ts. Add metadata to llm endpoint in main.py and pass to llm.py - you might have to edit models.py accordingly.

[ellipsis-dev[bot]]

To have BitBuilder create a Pull Request with the implementation, the user who created the issue (@nsbradford) can comment below with "LGTM". If the plan is inaccurate, edit the issue description and write a comment with "replan".

---

Implementation Steps

1. Add sessionId to the frontend
   • In the /ui/pages/index.tsx file, modify the handleNewUserPrompt function to include the sessionId in the sendLLMRequest function call. The new function call should look like this: sendLLMRequest({ model: 'gpt-3.5-turbo', messages: buildSummarizationPrompt(content, serverResponseMsg.results), sessionId: sessionId })
2. Modify sendLLMRequest function
   • In the /ui/shared/api.ts file, modify the sendLLMRequest function to include sessionId in the request payload. The new payload should look like this: { model: data.model, messages: data.messages, sessionId: data.sessionId }
3. Modify LLMChatCompletionRequest model
   • In the /backend/models.py file, modify the LLMChatCompletionRequest model to include sessionId as a new field. The updated model should look like this:

     class LLMChatCompletionRequest(BaseModel):
     model: str
     messages: List[LLMChatCompletionMessage]
     sessionId: str

4. Modify llm endpoint in main.py
   • In the /backend/main.py file, modify the llm endpoint to extract sessionId from the request payload. The updated endpoint should look like this:

     @app.post("/llm/")
     async def llm(request: LLMChatCompletionRequest):
     result = await llm_get(request.model, request.messages, request.sessionId)
     return { 'text': result }

5. Modify llm_get function in llm.py
   • In the /backend/llm.py file, modify the llm_get function to include sessionId as a parameter and in the metadata of the acompletion function call. The updated function should look like this:

     async def llm_get(model: str, messages: List[LLMChatCompletionMessage], sessionId: str) -> str:
     print(f"Calling LLM {model}")
     response = await acompletion(
     model=model,
     messages=[m.dict() for m in messages],
     temperature=0,
     metadata={"environment": getEnvironment(), "sessionId": sessionId},
     )
     text = response.choices[0].message.content
     print(f"LLM response: {response.choices[0].message.content}")
     return text

---

Generated with :heart: by www.bitbuilder.ai. Questions? Check out our the documentation.

[nsbradford]

lgtm