dark-peak-analytics / assertHE

R package to assist in the verification of health economic decision models.
https://dark-peak-analytics.github.io/assertHE/
Other
4 stars 10 forks source link

Add AI assist to describe a selected function. #56

Closed RobertASmith closed 7 months ago

RobertASmith commented 7 months ago

I've added a new function called summarise_function_with_LLM which is used to summarise a specific function that has been already sourced using GPT-3.5.

This is a trial run, and still early days. We will train our own LLM rather than use OpenAIs paid one.

try running this

rm(list = ls())

library(assertHE)
library(httr)
library(roxygen2)

devtools::load_all(".")

source_files(path = "tests/testthat/example_project/R",
             keep_source = TRUE)

response <- summarise_function_with_LLM(foo_name = "create_Markov_trace",
                                        llm_api_url = Sys.getenv("LLM_API_URL"),
                                        llm_api_key = Sys.getenv("LLM_API_KEY"))

response

The function requires three arguments: the function name, the llm url and key. The second and third default to system variables but can be overwritten with the url and key like this (fake key added for example):

llm_api_url = "https://api.openai.com/v1/chat/completions" llm_api_key = "sk-tSaEvcXc0Xvc0XoNagoNahTvc0XoNagg3BlbkFJoq"

These must be created on OpenAI. (First $5 free).

This will allow you to play with the function.

Next steps:

It would be great to merge the ability to see this in the shiny app into this branch. This could look like this:

image

image

The app should NOT allow more than 3 AI summaries in any one session.

W-Mohammed commented 7 months ago

Hi @RobertASmith,

Was this what you had in mind for the AI-assist feature in the shiny app?

https://github.com/dark-peak-analytics/assertHE/assets/58259938/0bf5236a-fc83-44f4-9fa9-f4ed00109887

I created a pull request for the AI-assist shiny implementation, #57.

W-Mohammed commented 7 months ago

@RobertASmith, three more things to highlight.

  1. The implementation of the AI-assist in Shiny did not guard for missing or invalid API keys.
  2. The app currently allows up to 5 chatGPT queries. I understand you only wanted three.
  3. The prompt needs more refinement. As you can see in the demo video, the AI response is not always in HTML.