Healthcare Questionnaire/Clipboard

[bevnobev]

Email Address* Ming.ligh@va.gov, kristen.mcconnell@adhocteam.us

VA Business Line / Department VFS Description of Use Case Questionnaire Product Outline MVP- Visit Intro Initiative Notifications Feature Email Appointment Confirmation -Create new email notification for the appointment scheduled that includes and unauthenticated (no PHI) link to clipboard that is due for the visit. Text Appointment Reminder - VeText - show link to clipboard that is due for the visit leveraging Appointment Reminders - show a link (not form specific) to the questionnaire manager letting them know they have a clipboard to complete Send a notification if the clipboard is not completed within X days of the appt. Link includes mockups of the text and email verbiage experience we are wanting • Additional storyboards for text and email • Content Approval on text notification

Notification Channel(s) SMS & Email

Estimated Volume (per month) 50 for pilot – expand beyond that after validation

Estimated Go-Live Date May 7th (Tentative)

Funding* Ming Ligh to provide - Funding Pending

Source Systems Va.gov, lighthouse PGD, VistA

Metrics Number of questionnaires completed via selection from link in email or text vs. locating on VA.gov

[bevnobev]

tagging @mlighdc and @kristenmcconnell

Mockup are here under the Presentation section..

[bevnobev]

Forgot to tag @pcshah1026 and @shanemelliott

[bevnobev]

@ffafara-tw A couple of questions came up in meeting"

• What kind of metrics can we track around the link in the email?
• Do we know if the email went to spam or the inbox?

[ffafara-tw]

What kind of metrics can we track around the link in the email?

When creating templates we can put Google Analytics tracking on a link for example: https://www.va.gov/find-locations/?facilityType=health&serviceType=Covid19Vaccine&utm_source=VANotify&utm_medium=email&utm_campaign=covid_19_vaccine

This allows you to see how much traffic is directed from the email to the webpage/system, however it does not provide per user tracking i.e. we don't provide information wether specific recipient clicked a link.

Providing information about clicks of specific users is technically possible. However there are 2 main issues with it:

• it requires usage of link forwarding/shortening service which often obfuscates link destination. A lot of providers use that as indication of spam. Moreover it can be misleading to recipients affecting their user experience and lowering overall trust.
• such tracking without explicit consent is a questionable practice (and illegal in Europe for example)

Do we know if the email went to spam or the inbox? Short answer: We do not (on a per user/email basis).

Longer answer: This really depends on email provider. We do get feedback from some providers (like AOL, Yahoo, Outlook) when a recipient marks email as spam. It generates an abuse complaint. At the moment that actually adds that email address to a suppression list preventing us from communicating with that email in the future. We also have a webhook that upstream systems can utilize to receive information when that happens.

If email is tagged as spam by the provider itself (not user) and delivered to spam folder there is no abuse complaint, so we do not get notified. We do have some visibility into aggregate spam statistics via SNDS and other mail reputation monitoring networks/tooling. For most providers these apply on IP/domain level.

Gmail is a big outlier. They mark messages as spam on individual recipient level based on past behavior. For example if recipient always deletes emails from VA without opening them, at some point Gmail will just move them automatically to the spam folder. We are in the process of registering for Gmail postmaster tool that will help us monitor our sender reputation with them.

We do have access to tools (like Litmus and AWS Delivery Dashboard) where we can test inbox placement of a specific campaign. This works by sending emails to test accounts monitored by these tools which can then report how the email was placed. If you are interested in such test please let us know.

[mlighdc]

@mdewey here are the questions we had regarding analytics about notifications (cc @leahkeeler)

[leahkeeler]

Bev and I connected today --> here are some questions that we need to address:

1. How will the pilot users be filtered so they get the updated template with clipboard url? 2.If Veteran takes action after first or second notification how will campaign terminate? ie what data is returned once questionnaire is filled out and submitted.

[miabecker]

Feels like we need to start with a dataflow diagram - can do in our next VEText meeting. @shanemelliott @pcshah1026 @ffafara-tw @bevnobev

[kristenmcconnell]

We are assigning questionnaires based upon a combination of the three following things:

• Facility
• Clinic
• Stop code (AKA the type of appointment)

Therefore, we know if a Veteran has a scheduled appointment at a particular facility and clinic with a certain appointment type, then that equals X type of questionnaire.

For our MVP, we only have one questionnaire, which will be for primary care patients seeing Dr. John Byrne at the Loma Linda VAMC.

[ken-non]

Hi all, update from the VEText team -

Text Appointment Reminder - VeText - show link to clipboard that is due for the visit leveraging Appointment Reminders - show a link (not form specific) to the questionnaire manager letting them know they have a clipboard to complete Send a notification if the clipboard is not completed within X days of the appt.

For a May 7th release, we can deliver text appointment reminders, it's a light lift without extra development work.

Email Appointment Confirmation -Create new email notification for the appointment scheduled that includes and unauthenticated (no PHI) link to clipboard that is due for the visit.

Wiring email reminders up to the text reminders is outside of what we've scoped so can't commit to it for May 7

Also, the clipboard / pvs integration work is in testing right now, so is on track for the release as well.

[bevnobev]

@mlighdc @leahkeeler making sure you are in the loop

[mlighdc]

@Kennon-git can you let us know what would be the timing for the email if it is not May 7th?

cc: @bevnobev

[kristenmcconnell]

Email wording:

You have a VA health questionnaire to complete.

Dear Hector Allen, Your VA provider sent you a brief online questionnaire to help prepare for your appointment on January 7, 2021 at Douglas VA Medical Center.

To fill out this questionnaire, go to https://www.va.gov/health-care/health-questionnaires/questionnaires/ and sign in.

Once logged in, you will be able to: • Tell your provider about your symptoms and other concerns • Review other completed questionnaires and print copies for your records

Thank you for your service, VA.gov

---

You are receiving this email because you have a scheduled medical appointment at a VA facility. Please do not reply to this email.

For questions about your upcoming appointment Please call your VA provider. Find your [provider’s phone number] (links to facility locator).

For questions about your health questionnaire Call our MyVA411 main information line at 800-698-2411, and select 0. We’re here 24/7. If you have hearing loss, call TTY: 711.

[kristenmcconnell]

Wireframe of the email.

[miabecker]

@mlighdc, just updating the issue for tracking purposes @pcshah1026 let us know that he connected with you and there is a meeting on Friday to discuss email timelines and VEText capacity. cc: @bevnobev

[mlighdc]

Hi @miabecker , I think we had to cancelled the meeting on Friday and is being rescheduled for Tuesday (5/11) at 1:30 pm ET. Is that correct?

[miabecker]

@mlighdc Kennon from VEText is meeting with Leah I believe about the Healthcare Questionnaire. Will attend if I can cc: @Kennon-git @bevnobev

[miabecker]

@kristenmcconnell @mlighdc looks like this content needs to go through a Privacy Review given the last name and the information about the health center they are going to. cc: @bevnobev @mlighdc

[mlighdc]

@kristenmcconnell I just submitted the email to Privacy for review. I asked if we can get feedback on all the items by the end of this week.

[mlighdc]

@miabecker we received feedback from Privacy and we will have to revised the email to remove PHI.

[miabecker]

@mlighdc thank you for the update!

[mlighdc]

@kristenmcconnell here is the general feedback from Privacy about the email:

• Remove the patient name
• Remove any reference to the type of appointment, such as medical
• Remove reference to provider
• Can keep date and place of the appointment
• Should use a vanity URL so they do know the type of appointment or that it is health-related

Maybe the header can be You have a questionnaire to complete before your appointment.

cc - @DanielleThierryUSDSVA

[miabecker]

@mlighdc Wanted to share our experience a bit we have gotten in sync with POs previous that first name is ok, but not last name. I am curious about removing reference to "medical" because if its clear that the email is coming from VHA then that would already communicate that fact. We have seen though that details about the specific type of appointment i.e. neurology are not allowed. Provider I want to do some more research on for you because we seen that from other health organizations. Curious who your PO was?

[mlighdc]

@miabecker The PO is Stephania Griffin. That would be great if we can add the first name. She did suggest that make the appointment type generic.

[kristenmcconnell]

I do worry about removing provider too. During research, Veterans discussed how they probably wouldn't click the link unless they knew that this was a request from their provider.

[miabecker]

Just an update I don't know if this will get done in time for this use case but I am going to try and work on drafting some guidelines that can get approved by privacy officers so that way we don't run into these sorts of issues i.e. being able to have the first name.

[mlighdc]

That would be great @miabecker! Thank you!

I am cc @batemapf. Patrick is taking over the healthcare questionnaire and the broader initiative of the check in experience (which the healthcare questionnaire is merged into).

[miabecker]

Pending VEText work to get this template live. VANotify needs to create template on our side.

[miabecker]

Clipboard project is on hold currently. Not currently in the appointment reminder due to negative feedback from Veterans during usability testing. Pending LRODA (low-risk identity management) project - sending URL, with UID to VA.gov so folks can log in to fill out a questionnaire.