Business intake form for Check In Experience

[randomsync]

Your Details

1. Gaurav Gupta ​

2. Business Line health ​

   Notification Details

3. Notification type text/SMS ​

4. Please describe your use case. We'd like to submit a travel claim as part of the check-in process, and would like to send them a text message with the status of the claim submission. ​

5. What actions can the user take based on the notification they receive? They can login to their profile and view the status of their travel claim submission ​

6. What is the desired business outcome? Ability for Veterans to file travel claim conveniently during the online check-in process. ​

7. What system will kick off the notification? Please note the system should be inside the VA or have an ATO. vets-api ​

8. What will trigger the notification? The notification will be triggered when veterans select to submit a travel claim during the online check-in process ​

9. Provide sample content per notification type, if you have it. ​

10. Has a Privacy Officer (PO) seen and approved the content? If not, do you know who your PO is? No. No (i'll check with project lead) ​

11. Would you prefer to provide contact information for the recipients or would you rather VA Notify look this up for you by Veteran ID? If by ID, please let us know what identifier is used in your system. We'd like VA Notify to look this up by ICN ​

12. Do you currently capture communication preferences related to this notification? If so, please describe. We do NOT capture communication preferences. ​

13. What is the anticipated volume of notifications per day, week, month? ​ Based on our current usage, we see about 1500 check-ins per day. We don't have accurate data on what % of those will file travel claim, but can go with the upper bound of 1500/day for now. Also note: this volume will increase in future as more Veterans use the online check-in method.

14. When does this notification need to be in production? We're currently working on the designing/implementation on our side, and are estimating to be ready to test by end of April, with production release right after.

[mjones-oddball]

@benbrasso-agile6 Are you planning to include any verbiage on the PCI app to indicate that a text message will be sent once the claim is "processed" (for lack of a better term)? It may be something we could add there to just indicate that the Veteran will receive text messages for claims submitted through this appointment check in process, however, if anything goes wrong or needs to be submitted after this time they should go to BTSSS and complete all further actions, as well as status checks, there.

[mjones-oddball]

@randomsync PCI Travel Claim Confirmation staging template_id: 0e0dd0c8-b482-4055-8c96-22aa684022ce

We received your travel reimbursement claim for your VA appt on ((appt_date)). Claim number, ((claim_number)). Status updates will be available online.

PCI Travel Claim Error staging template_id: 9cb33a3a-1264-4a14-b7e8-d7b5e955c40d

We're sorry, something went wrong with your travel reimbursement claim. Try again within 30 days of your appointment, http://va.gov/accessva-travel-claim

PCI Travel Claim Duplicate staging template_id: 8cbdbeab-69f1-421b-b035-e303c40bdef8

A travel reimbursement claim was already submitted for this appointment. Check the status, http://va.gov/accessva-travel-claim

Please note, to send personalisations, e.g. appt_date, in your notification API request you will include a personalisation object with all fields included in the notification and their associated values. See example body below.

[benbrasso-agile6]

@benbrasso-agile6 Are you planning to include any verbiage on the PCI app to indicate that a text message will be sent once the claim is "processed" (for lack of a better term)? It may be something we could add there to just indicate that the Veteran will receive text messages for claims submitted through this appointment check in process, however, if anything goes wrong or needs to be submitted after this time they should go to BTSSS and complete all further actions, as well as status checks, there.

Yes. Confirmation page.

Our "Need help?" section at the bottom of the page addresses your latter point.

[kellysmith1008]

@mjones-oddball We have updated our text messages for Travel Pay per feedback received from our CAIA review. Mural with revisions

[benbrasso-agile6]

@mjones-oddball our team is set to begin QA in our staging environment now. I wanted to get a sense of timing with you on some outstanding items:

• Updating the templates based on Kelly's message

@mjones-oddball We have updated our text messages for Travel Pay per feedback received from our CAIA review.

• Opt out messaging

As a note, we may have to include some standard opt out messaging at the end, which I'll work with VEText on.

Thank you!

[mjones-oddball]

Hey @benbrasso-agile6 Some key folks are away at an offsite through the end of this week. I will be syncing with VEText on Tuesday to review your updated messaging and will also discuss the possibility of opt out messaging. I'll be in touch with updates after that meeting.

[benbrasso-agile6]

One thing that just crossed my mind, if we include an opt out, and someone does opt out, we have no way of letting them know when a claim was successfully submitted to BTSSS or not.

[mjones-oddball]

@benbrasso-agile6 That's a fair concern. By law we have to honor SMS opt outs, so it's not a matter of "if" that has to happen, but rather how can we ensure the recipient opting out of travel claim messages doesn't impact other use cases shared by that phone number they may want.

Something to consider is that you can also send an email - which is typically the notification channel I would use for confirmations or notifications that you expect the recipient to refer back to later. I could get you set up with your own service to do that outside of VEText. It's just another API request, and this time we could take the ICN to look up the email address in VA Profile because email isn't something they are providing as part of PCI (right?). It could also be an opportunity to send them a summary of their check in experience (without including PII) and/or give them more detailed instructions on how to submit travel reimbursement claims if they didn't do so through the check in experience. That could be different email templates/scenarios. Just depends on what you want to do.

[benbrasso-agile6]

Thanks. I think we need to solve for making sure the Veteran sees the notification asap, e.g., before they walk out of the facility. For example, if they know that the claim didn't go through, then they have an opportunity to stop by the travel office before leaving (if they're at a facility with a travel office). But, I think it's something our team will want to discuss a bit more with @mdewey. I'm going to schedule some time for our team to discuss this a bit further.

[mjones-oddball]

Just keep in mind, the ability to opt out doesn't mean a Veteran actually will. Especially knowing that they purposefully kicked off the PCI process by texting in the first place. It's just something we have to account for.

[mdewey]

Relevant slack thread https://dsva.slack.com/archives/C02G6AB3ZRS/p1683302022027119

Summary:

• A Veteran cannot opt out of Check-In messages and, by extension, this message since it's Veteran initiated.
• The worse case is that the Veteran starts check-in 45 minutes before and has till 15 minutes after their appointment to check in (and, by extension, submit a claim), so its at worst 1 hr from initiation to claim submission message
• We are tabling questions and such till after a VeText meeting happening on 5/9

[mjones-oddball]

Summarizing some findings here from slack discussion:

• No one can come back to the PCI app once they go through it.
• Technically someone could start the PCI process by texting the phone number and then not click on the check in link for the full hour (it's open 45 minutes prior and 15 mins after appt time), but probably not typical.
• The API for processing the claim takes up to 45 seconds, so they are not posting the confirmation (success/failure) in app. See below.

  As a Veteran, If I have a simple Travel Pay claim, I can submit my claim when I check in for my appointment. The initial MVP for this use case had that call inline, as you suggested, but the 45 sec wait time was unusable during testing. Since this is a mobile-only experience happening during the Veteran Checking in, that wait time is unacceptable. The API call is not able to be made faster (yet, hopefully, by fall, its better, but that's just a hope), So in order to not have all the feedback be the app is slow and unresponsive, we pivoted to asynchronous model using SMS as the feedback and let the Veteran check into their appointment instead of waiting for the claim to be submitted

[mdewey]

Hey @mjones-oddball!

I wanted to reach back out. Our testing is in staging is almost complete and we are to eye ball production. We are looking to ideally release early to mid June. I know that templates are needed to be made in production. What are the next steps in getting those created and what API keys do we need to have created as well?

[mjones-oddball]

@mdewey @loripusey @randomsync @benbrasso-agile6
Hi all! We met with VEText and came to the following conclusions:

• No opt-out keywords will be added to the templates for this use case. We're going to accept the risk because it's highly unlikely that someone would initiate PCI, go through the process, submit a claim, receive a text within 45 seconds of submission, and then ask to opt-out (max 1 hour duration for the entire process). This was the same logic from the original design, we just wanted to confirm it still made sense given the new addition to the process.
• Your error use case text is slightly too long (by 4 characters). You could 1) remove the word "online" or 2) work with va.gov on a shorter url/redirect for the how to file link. Which do you prefer?

We're sorry, something went wrong. File your travel claim online within 30 days at https://www.va.gov/resources/how-to-file-a-va-travel-reimbursement-claim-online/

• Can you confirm your content was approved by a Privacy Officer?
• Before we move on to production, can you please provide a test user so Shane can see it working end to end. Alternatively you could schedule an end to end demo if you'd prefer to do that.
• In the meantime, I will get your new assets prepared so I can provide you with the configs when you have sign off from Shane.

CC @shanemelliott @bevnobev

[kellysmith1008]

Hi @mjones-oddball and @benbrasso-agile6 In the CAIA ticket, the proposed text message for the error text is "We're sorry. We couldn't process your travel pay claim. File your claim online within 30 days at https://www.va.gov/health-care/get-reimbursed-for-travel-pay" which comes in at 157 characters. Is this acceptable to you both?

[mjones-oddball]

@kellysmith1008 It looks like the content has changed slightly in the Mural since I last copied them. Can you please post here for clarity the final desired content for each use case: success, duplicate claim, and error? For example, the duplicate claim content in the mural now says Jun 8 at 1:00, but before it just had a date in format: MM/DD/YYYY.

[kellysmith1008]

Hi @mjones-oddball Yes, for sure. We received feedback from CAIA on the text messages and updated them based on their feedback. I'll post them here for the current suggestions:

Success:

• We received your travel pay claim for SEPT 30. Your claim number is TC202207000011666. Check your claim status at https://va.gov/accessva-travel-claim

Existing claim:

• You already filed a travel pay claim for your appointment on JUN 8 at 1:00. Check your claim status at https://va.gov/accessva-travel-claim

Error message:

• We're sorry. We couldn't process your travel pay claim. File your claim online within 30 days at https://www.va.gov/health-care/get-reimbursed-for-travel-pay

[mjones-oddball]

@benbrasso-agile6 In the existing claim use case. Is the appt date/time you're including in the applicable timezone of the VAMC location?

[benbrasso-agile6]

We're not moving forward with the time parameter. Here is the most up to date content.

SMS Messages for Travel Reimbursement Claims.docx

[mjones-oddball]

@benbrasso-agile6 thank you! Can you let me know when you get your PO signoff?

[benbrasso-agile6]

Yes, I will. Haven't heard back yet.

[benbrasso-agile6]

Hey @mjones-oddball, couple updates and questions:

1. Still waiting to hear back from PO
2. Have you had a chance to update the templates based on my previous comment?

We're not moving forward with the time parameter. Here is the most up to date content.

SMS Messages for Travel Reimbursement Claims.docx

3. Does VA Notify provide any reporting or access to reporting around messages sent, delivered, failed, etc.?

Thank you!

[benbrasso-agile6]

And, @mjones-oddball please remove extra spacing in the messages. For example, there's two spaces before the claim number.

[mjones-oddball]

@benbrasso-agile6 I updated the success and duplicate claim templates.

1. The error template is a little too long (173 characters).

We’re sorry. We couldn’t process your travel pay claim. File your claim online within 30 days at https://va.gov/resources/how-to-file-a-va-travel-reimbursement-claim-online/

@kellysmith1008 had suggested the alternate content below (157 characters)

We're sorry. We couldn't process your travel pay claim. File your claim online within 30 days at https://www.va.gov/health-care/get-reimbursed-for-travel-pay

Thoughts?

Does VA Notify provide any reporting or access to reporting around messages sent, delivered, failed, etc.?

2. VA Notify offers aggregated metrics in Domo. Do you have access to Domo? You should have an extremely low (to 0) failure rate post-launch because you will be sending notifications to Veterans who initiated check in using the exact phone number they texted from. Example below.

[benbrasso-agile6]

Yes, let’s use that version. Thank you!

[benbrasso-agile6]

No, we typically use DataDog. I’d have to check Platform documentation on how to access Domo. Is there (or will there be) a specific report in Domo for us to see data for our notifications?

[mjones-oddball]

@benbrasso-agile6 error template updated. Uploading a revised version of your "final templates" doc. SMS.Messages.for.Travel.Reimbursement.Claims.docx

You can use PCI's datadog account to track your API request successes and failures to the VA Notify API. But "success" at that point just means you have successfully sent us a notification request. After that we reach out to Twilio to send the message and would retry if needed (e.g. let's say Twilio is temporarily unavailable). Once it's delivered (or fails to deliver) we receive a status update from Twilio (which is also sent to VEText since it is their service and phone number). Each day around 10:15 am ET Domo will update with the latest data from the prior day. All live notifications appear in VA Notify's Domo dashboard but you can filter by your template names and adjust things like the date range.

Template names:

[benbrasso-agile6]

Thanks. I’ll work on getting access.

[mjones-oddball]

https://depo-platform-documentation.scrollhelp.site/analytics-monitoring/analytics-customer-support-guide#Analyticscustomersupportguide-2.RequestaccesstoDomo

[benbrasso-agile6]

I see the VA Notify dashboard. Assume we won't be able to see the PCI filters until we get some data in there.

[mjones-oddball]

That's correct

[mjones-oddball]

@benbrasso-agile6 @randomsync @kanchanasuriya I've created your templates in production. Please find your Staging and Production configurations in this Sharepoint document. If there's any other reference info you want to add feel free to edit, just not the api keys please. CC @shanemelliott @bevnobev

[kanchanasuriya]

Thank you @mjones-oddball. We got the template IDs from the shared doc. Could you also please share the api keys for the production environment? Please send it to kanchanadevi.suriyamoorthy@va.gov

[mjones-oddball]

@kanchanasuriya Sent to your va.gov email

[mjones-oddball]

@benbrasso-agile6 @kanchanasuriya @randomsync Hey all, just checking in. What is the latest status? Are we still awaiting PO approval of the message content? How is testing going?

[benbrasso-agile6]

Re: PO review, we have emailed Stephania 3 times with no response. We're not quite sure how to proceed at this point.

June 20 is our targeted release to production date. Although, I believe we are going to release sooner to test in production with extremely limited access.

[bevnobev]

@benbrasso-agile6 Stephania responded to Mark and had concerns around claim number

[benbrasso-agile6]

@bevnobev thanks, I'll talk to Mark

[mdewey]

Yes there are some concerns around the the Claim number as PII in the text message.

We are currently looking at if/what changes need to happen and when.

[bevnobev]

Is putting the last 4 digits of the claim number helpful to the Veteran? That may suffice the privacy office

[loripusey]

@mdewey Since our pilot is starting in 2 weeks, should I assume that we should leave the Claim Number out until this discussion is resolved? cc: @benbrasso-agile6 @brianseek

[mdewey]

To update the ticket:

We have landed on the updated the text to the following:

We received your travel pay claim for SEPT 30. You can check the status of your claim ending in 1666 at https://va.gov/accessva-travel-claim.

There are minor updates to the second sentence and we are only sending the last 4 of the claim number.

cc: @mjones-oddball @benbrasso-agile6 @loripusey

[mjones-oddball]

@mdewey @benbrasso-agile6 @loripusey I've updated the SMS templates in Staging and Production accordingly! Thanks for the update. I see you mentioned a launch date soon. Do you have a target day/timeframe to share?

[loripusey]

Our pilot target date is June 20.

On Fri, Jun 9, 2023 at 12:22 PM Melanie Jones @.***> wrote:

@mdewey https://github.com/mdewey @benbrasso-agile6 https://github.com/benbrasso-agile6 @loripusey https://github.com/loripusey I've updated the SMS templates in Staging and Production accordingly! Thanks for the update. I see you mentioned a launch date soon. Do you have a target day/timeframe to share?

— Reply to this email directly, view it on GitHub https://github.com/department-of-veterans-affairs/va.gov-team/issues/55526#issuecomment-1584839304, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUVJZVRDY3NYKZCOHS4RW6TXKNETPANCNFSM6AAAAAAWC665VM . You are receiving this because you were mentioned.Message ID: @.*** .com>

-- Warmly, Lori Pusey - VFS/Modernized Check-in - Product Manager [Agile Six]

[kellysmith1008]

@mjones-oddball I think this may already be updated, but we found a fun trick with iOS text messages that display a link preview instead of the link itself when a user has saved "53079" as a contact in their iphone. To get around this, we need to add a period to the end of the URLs for all 3 text messages. I've updated them in the SMS message system so you can use those for reference. Thank you! cc @benbrasso-agile6 @loripusey

[mjones-oddball]

@kellysmith1008 I've added a period to the end of the duplicate and error messages. Confirmation already had it.

[randomsync]

@kanchanasuriya Sent to your va.gov email

@mjones-oddball can you please send the api key and service id for production to my email as well: gaurav.gupta@va.gov

[mjones-oddball]

@randomsync sent

[mjones-oddball]

@randomsync @loripusey @benbrasso-agile6 Did the pilot launch yesterday and if so how did it go? Just checking in.

[benbrasso-agile6]

Moved to this coming Monday