search

FDA-AI / FDAi

Discovering the positive and negative effects of every food and drug in the world!
https://fdai.earth
20 stars 19 forks source link

Voice2Measurements Library/API/Client for Robocall Service for Data Collection and Reminders #85

Open mikepsinn opened 1 year ago

mikepsinn commented 1 year ago

Technical Documentation: Creating an AI Agent for Daily Patient Interactions and Data Collection

Overview: This document outlines the workflow and implementation details for creating an AI agent that interacts with patients daily, collects health-related data, structures it as time series measurements, and stores it in a database using the OpenAI API.

Workflow:

  1. Daily Interaction Setup:

    • The AI agent is scheduled to initiate daily interactions with patients through their preferred communication channel (e.g., SMS, email, or a dedicated app).
    • Each interaction focuses on collecting specific data on symptoms, dietary intake, supplements, and medications.

  2. Data Collection:

    • Patients are prompted to report their intake of foods, supplements, and medications, including specific amounts and units.
    • The AI agent asks follow-up questions to ensure completeness and accuracy, including checking for any items the patient may have forgotten.

  3. Data Structuring:

    • Collected data is structured as time series measurements, with each entry tagged with a timestamp, the type of data (food, supplement, medication), quantity, and unit.
    • This structure facilitates tracking changes over time and performing causal inference analysis.

  4. Data Storage:

    • Structured data is stored in a secure, HIPAA-compliant database.
    • The database design supports scalability, efficient querying, and data privacy.

Implementation Details:

Conclusion: Creating an AI agent for daily patient interactions and data collection involves setting up a reliable interaction schedule, employing natural language processing for data collection, structuring the data for analysis, and ensuring secure data storage. This system enables personalized health recommendations and supports ongoing health management.

Example Conversation

FDAi Agent: "Good morning! Let's do our daily check-in. Can you provide today's intake of specific foods, supplements, and any medications, including doses and units?"

Patient: "Sure. For breakfast, I had eggs and white bread, and for lunch, fried chicken with french fries. I haven't taken my supplements yet, but I plan to take vitamin D and omega-3. I'm on Fluoxetine for depression."

FDAi Agent: "Thank you. How much vitamin D and omega-3 will you take, and what's the dosage of Fluoxetine?"

Patient: "I take 1000 IU of vitamin D, 1000 mg of omega-3, and 20 mg of Fluoxetine."

FDAi Agent: "Let's optimize that. Based on your conditions and intake:

Forgotten items? Any other foods, medications, or supplements not mentioned?

Supplement efficacy: You've been on vitamin D and omega-3 for 3 months with minimal improvement in psoriasis severity. Considering the expense—approximately $30 monthly on these supplements—it might be wise to reallocate funds towards higher doses or different supplements, like turmeric (curcumin), which has shown benefits in inflammation and depression at 500 mg daily.

We'll also track zeros for non-taken items for accurate causal inference. It's crucial to monitor onset delays and action durations, adjusting our strategy as we gather more data."

This approach ensures daily, precise tracking of the patient's intake and symptoms, allowing for data-driven adjustments and recommendations tailored to their specific health conditions and lifestyle.

User Story:

When I visit my grandma, I try to collect data on her cognitive performance and factors that could influence them.

I also talk to her for a while. But she can only remember 5 minutes, so I repeat myself 12 times in an hour.

She'd probably like to talk to me 16 hours a day, but I'd get even less done than I already do.

So it would be cool to have a robocaller service that calls lonely old people, reminds them to take their grandma, and talks to her for about 16 hours.

Also, I'd like it if I got a phone call every day and could do my treatment/diet/symptom tracking that way. i.e.

Then it would convert the responses to measurement objects and post to whatever endpoint is specified.

Example Measurement Array for the response, I took 5 grams of NMN:

[
    {
        "combinationOperation" : "SUM",
        "startAt" : "{ISO_DATETIME_IN_UTC}",
        "unitName" : "grams",
        "value" : "5",
        "variableCategoryName" : "Treatments",
        "variableName" : "NMN",
        "note" : "{MAYBE_THE_ORIGINAL_STATEMENT_FOR_REFERENCE}"
    }
]

Nice possible future feature: Since people with Alzheimer's don't remember what you said before, ideally, it could eventually use verbal and maybe frequency data to quantify how nice each statement makes her feel. Then, it could gradually say more of the nice things that make her the happiest since it's usually like she heard it the first time.

Draft Implementation Roadmap

For reference and cannibalization, I implemented statement intent identification and handling in PHP and JavaScript here https://github.com/search?q=repo%3AFDA-AI%2FFDAi%20intent&type=code

It's dumb relative to what's possible with LLM's now, though:

We may want a T3 framework API or something less monolithic and more maintainable.

Vocode also has a great framework for this. The response times in the demo are superfast. https://docs.vocode.dev/welcome

Milestone 1 - Design service architecture

Milestone 2 - Build core platform

Milestone 3 - Develop health-tracking features

Milestone 4 - Add cognitive assessment and talk therapy

Milestone 5 - Enhance personalization

Milestone 6 - Launch and iterate

mikepsinn commented 12 months ago

Here's a SYSTEM prompt that seems to work

You are a robot designed to collect diet, treatment, and symptom data from the user.  

Immediately begin asking the user the following questions
- What'd you eat today?
- What'd you drink today?
- What treatments did you take today?
- Rate all your symptoms on a scale of 1 to 5.

Convert the responses to the following JSON format
[
    {
        "combinationOperation" : "SUM",
        "startAt" : "{ISO_DATETIME_IN_UTC}",
        "unitName" : "grams",
        "value" : "5",
        "variableCategoryName" : "Treatments",
        "variableName" : "NMN",
        "note" : "{MAYBE_THE_ORIGINAL_STATEMENT_FOR_REFERENCE}"
    }
]

That would be the result if they said, "I took 5 grams of NMN."

For ratings, use the unit `/5`.  The `unitName` should never be an empty string. 

Also, after asking each question and getting a response, check if there's anything else the user want to add to the first question response. For instance, after getting a response to "What'd you eat today?", your next question should be, "Did you eat anything else today?".  If they respond in the negative, move on to the next question. 

Your responses should be in JSON format and have 2 properties called data and message.  The message property should contain the message to the user.  The data property should contain an array of measurement objects created from the last user response.
mikepsinn commented 5 months ago

You can try out the demo here https://www.vocode.dev

mikepsinn commented 5 months ago

Here's how you can send a measurement with the demo token with python:

import requests

url = "https://app.quantimo.do/api/v3/measurements/post?userId=1"

payload = [
    {
        "sourceName": "FDAi",
        "unitAbbreviatedName": "count",
        "variableCategoryName": "Foods",
        "variableName": "Apple",
        "combinationOperation": "SUM",
        "measurementItems": [
            {
                "value": 1,
                "timestamp": 1707008151
            }
        ]
    }
]
headers = {
    "accept": "application/json",
    "content-type": "application/json",
    "authorization": "Bearer demo"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

and javascript


const options = {
  method: 'POST',
  headers: {
    accept: 'application/json',
    'content-type': 'application/json',
    authorization: 'Bearer demo'
  },
  body: JSON.stringify([
    {
      sourceName: 'FDAi',
      unitAbbreviatedName: 'count',
      variableCategoryName: 'Foods',
      variableName: 'Apple',
      combinationOperation: 'SUM',
      measurementItems: [{value: 1, timestamp: 1707008151}]
    }
  ])
};

fetch('https://app.quantimo.do/api/v3/measurements/post?userId=1', options)
  .then(response => response.json())
  .then(response => console.log(response))
  .catch(err => console.error(err));
mikepsinn commented 5 months ago

Here's the text-to-measurements library that could be used to convert the text to structured measurements

https://github.com/wishonia/FDAi/blob/develop/libs/text-2-measurements/src/lib/statement-2-measurements.spec.ts

mikepsinn commented 4 months ago

https://www.retellai.com

mikepsinn commented 2 months ago

Started in #195