[discovery] NOD | Notification capabilities, deadline and decision data

[saderagsdale]

Value Statement

As a developer I want to know what our options are for 1) letting Veterans know the exact decision date of their decision letters, 2) sending notifications to remind Veterans of AMA deadlines, and 3) sending Veterans their decision letters via email. So that we can make it easier for Veterans to submit a NOD in a timely and efficient manner.

---

Background Context

The team just finished high-level brainstorming and prioritization for NOD, and would like to explore the following use cases:

1. A Veteran wants to know the exact decision date when adding a new issue.
2. A Veteran wants to know when their filing deadline is approaching so they can take action.
3. A Veteran wants to have to have their decision letter delivered via email, because decision letters don't always arrive in the mail.

Out of scope

• Actual code work

Open questions

• What are our development options for these use cases?

Tasks

Use Case 1: A Veteran wants to know the exact decision date when adding a new issue.

• [ ] Figure out how to supply Veterans with the exact decision date of their decisions.

Use Case 2: Veteran wants to know when their filing deadline is approaching so they can take action.

• [ ] Figure out how we might send email and SMS notifications to Veterans.
• [ ] Figure out how we might schedule such notifications to be run at a future date.
• [ ] Figure out whether we are considering sending Veterans multiple reminder notifications, per appeal, and if so, how we might schedule multiple reminder notifications to be run at future dates.
• [ ] Figure out how to supply our scheduled notifications with AMA deadlines.

Use Case 3: A Veteran wants to have to have their decision letter delivered via email, because decision letters don't always arrive in the mail.

• [ ] Figure out whether we can send decision letters to Veterans via email

Other Tasks:

• [ ] Huddle with team to refine options
• [ ] Talk to Matt and Steve about approach

Definition of Ready

• [ ] We're aligned on what we're investigating

Definition of Done

• [ ] We have a reasonable understanding of what our options are and how each solution would be implemented
• [ ] Options are written into the comments of this ticket
• [ ] We have buy-in from a stakeholder perspective
• [ ] Reviewed and approved by product and/or design

[rubensun]

Some materials that may be relevant to this ticket

• VA Notify Documentation
• Current list of notifications cataloged by VA Notify (Source: Google Docs)
• List of Form-related Notifications
• Research relating to effectiveness and timing of delivery
• Onsite Notification Documentation *

* Currently on hold in the short term see VANotify/onsite notifications strategy

[data-doge]

Here are my notes so far, per use case, and per task. @kylesoskin feel free to edit this comment (I think you should be able to edit it?)

Use Case 1: A Veteran wants to know the exact decision date when adding a new issue.

Context: In the NOD form, there is an "add issue" page (pictured below), where a Veteran can manually add an issue that was not present in the list of Lighthouse-provided contestable issues on the previous page. From what I understand, the Veteran's issue could be absent from the previous page because: 1) there was an error with Lighthouse contestable_issues endpoint that we use to fetch issues or 2) Lighthouse doesn't have the issue in their system yet. It may also be possible that the contestable_issues endpoint we're using is only returning disability-related issues.

Task 1: Figure out how to supply Veterans with the exact decision date of their decisions.

If a Veteran is adding an issue manually, they need to know the exact decision date of the decision on their issue. The first question is whether we could expect the contestable_issues, when working, to return a full list of disability and non-disability issues. If so, then we might consider modifying the UX around contestable_issues endpoint failures (see related: https://github.com/department-of-veterans-affairs/va.gov-team/issues/62872), to encourage the veteran to come back and try again later, rather than manually add their issue. Just a thought. But if it turned out that the contestable_issues endpoint, when working, could only return disability-related issues, then we would have to look elsewhere for non-disability-related issues. We might be able to get that information from the claim status tool, in which case we could link Veterans to it. But if it turned out that the claim status tool had all the data we needed, then we would want to rely on it, instead of the contestable_issues endpoint, alone, on the issues page in NOD. It might make sense to always include a link to the claim status tool on the "add issue" page to handle cases where the contestable_issues endpoint isn't working.

Next Steps

• [x] Figure out whether the contestable_issues endpoint returns all issues, or just disability-related issues.
• [x] If the latter, figure out whether the claim status tool and its endpoints returns all issues (or at the very least, more than just disability-related issues).
  • Claim status tools' endpoints return all issues, but not their decision dates. Jerek says we might be able to get that from BGS, via the bgs_ext gem.
• [x] If the claim status tool and its endpoints don't return anything more than the contestable_issues endpoint, briefly investigate whether alternative data sources exist.

Addition from Kyle:

<kyle> Figure out whether thecontestable_issuesendpoint returns all issues, or just disability-related issues. It can return

It can return them all (or at least more than just disability, I am not sure if it is exhaustive of all types of issues), but not all in 1 request, each must be asked for individually.

{
    "errors": [
        {
            "title": "Unprocessable Entity",
            "code": "unprocessable_entity",
            "detail": "benefit_type must be one of: compensation, pensionSurvivorsBenefits, fiduciary, lifeInsurance, veteransHealthAdministration, veteranReadinessAndEmployment, loanGuaranty, education, nationalCemeteryAdministration",
            "status": "422"
        }
    ]
}

The request we currently make is for benefit_type=compensation. I am not sure what all that covers and/or what additional ones we need to call/support for HLR? All of them? I have no idea that is probably a research question. </kyle>

Use Case 2: Veteran wants to know when their filing deadline is approaching so they can take action.

Context: After a decision is made on a claim, Veterans have 90 days to file an NOD. We want to send email and SMS reminders about this deadline.**

Task 2: Figure out how we might send email and SMS notifications to Veterans.

VANotify appears to be the way to send email, SMS, (and push), notifications to Veterans. They have a lot of great documentation. Good reading:

• VANotify Playbook (Tells you how to set up VANotify for your application)
• VANotify Product README
• VANotify Notification Guide
• VANotify 2023 Roadmap

To use VANotify, we would follow the steps in the playbook linked above. Notably, VANotify's 2023 roadmap contemplates "add[ing] support for Twilio notification status callbacks," that I presume would allow us to handle failed SMS deliveries. An open question I have is whether we would need VANotify to obtain a new short-code phone number for us. I'm guessing we could just use the same short-code phone number that I presume is shared by all of VA.gov. If we couldn't, then it could take up to 3 months to obtain a shortcode phone number.

Next Steps

• [ ] Check whether VANotify Twilio notification status callbacks have been set up yet.
• [ ] Check whether we would need to obtain a new short-code phone number if were to use VANotify to set up NOD reminders.

Addition from Kyle:

<kyle>

I think both these are premature. We know we can use VA notify in one way or another to contact the veteran. The context of this point, or the first overview of it is:

After a decision is made on a claim, Veterans have 90 days to file an NOD. We want to send email and SMS reminders about this deadline.**

We need to first figure out if and how we know what the deadline is to even be able to notify the vet at that point. Figuring out how we contact them before figuring out if we can even know when we need to contact them seems like getting ahead of ourselves. We know VA notify can be used in one way or another. We do not know that we can easily or reliably figure out when deadlines are (and therefore, 90 days from them). </kyle>

Task 3: Figure out how we might schedule such notifications to be run at a future date.

Reminders need to be scheduled, so this is something we need to figure out. This is might be pretty tricky. Here are some options:

Option 1: VA Event Bus

There is a team that is currently working on a VA Event Bus that would receive events from VBMS (including, potentially, "Decision Letter Created" events) and publish them to subscribers. Ideally, we could subscribe to any events relating to the creation/filing/delivery of a decision letter, and upon receiving such an event and when appropriate, enqueue a Sidekiq job to send an email and SMS message to the Veteran. This solution would depend on a number of factors, including:

• Whether the VA Event Bus is ready to use
• Whether it would receive the events we need from VBMS
• Whether we would be able to subscribe and filter down to only the events relevant to our use case.
• Whether the event would contain all the information we need to send our notifications (e.g. decision date; issue name; Veteran's phone and email, or the means to programmatically obtain the Veteran's phone and email)

Option 2: ???

Next Steps

• [ ] Check on the status of VA Event Bus.
• [ ] Figure out whether VA Event Bus, whenever it is available, could give us the information we need for our use case.
• [ ] Think about alternatives.

Additions from Kyle

<kyle> In contact with Emily Wilson who is an engineer on the event bus team. Will update with things when I get more info.

These rest of these seem like things to figure out later and not tack on to this already massive spike. All seem premature. Only other note was that we can remind vets easily about In Progress Forms about to expire already, just have to work with VA Notify to enable/turn it on.

</kyle>

Task 4: Figure out whether we are considering sending Veterans multiple reminder notifications, per appeal, and if so, how we might schedule multiple reminder notifications to be run at future dates.

TODO

Next Steps

TODO

Task 5: Figure out how to supply our scheduled notifications with AMA deadlines.

TODO

Next Steps

TODO

Use Case 3: A Veteran wants to have to have their decision letter delivered via email, because decision letters don't always arrive in the mail.

Task 6: Figure out whether we can send decision letters to Veterans via email

(Rocio says she already gets decision letters via email)

[saderagsdale]

@data-doge review and let us know when you're ready to chat.

[data-doge]

@saderagsdale I haven't had a chance to review this ticket yet, but it's on my todo list

[data-doge]

Quick notes for tomorrow's conversation with Eileen and Sade:

Use Case 1:

• We can get an approximate decision date (approxDecisionDate) from the contestable issues returned by Lighthouse's contestable_issues endpoint. But we need to figure out how approximate this date is, and whether it would warrant a disclaimer.
• Currently, we are calling Lighthouse's contestable_issues endpoint with a benefit_type of compensation. We need to figure out whether this endpoint returns the contestable issues that we expect the NOD form to encompass. That means we need to figure out exactly what benefits the NOD form should and should not encompass, and what Lighthouse benefit_types they map to. Depending on what we learn, we may need to call contestable_issues multiple times, with different benefit_types.

Use Case 2:

• The notification work is going to be very complicated and time-consuming. We most likely need an event bus. There is a VA team working on building an event bus, but they are too early in their development for us to consider them as an option. We could try building our own custom solution, but we likely won't have the time and resources for that.
• We can, however, set up notifications to remind veterans to complete in progress forms. VANotify has an infrastructure for that, that we can opt in to.

Use Case 3:

• Rocio says that she already receives decision letters via email. Rocio remembers opting in to email notifications sometime in the past few months.
• Robin says that he only receives decision letters via snail mail. He thinks that to receive email notifications, he might have to configure notifications in his profile.

[data-doge]

Follow-ups from our conversation on August 24, 2023:

• [ ] Figure out how approximate the approxDecisionDate is. Depending on how approximate it is, we may need to offer a disclaimer on the AddIssues screen and in any deadline notifications. To answer this question we can ask Lighthouse and Emily. We should also get some examples of real responses, possibly from vets on our team.
  • Asked Emily here: https://dsva.slack.com/archives/C5AGLBNRK/p1693252476948299?thread_ts=1693251475.285339&cid=C5AGLBNRK.
  • Asked Lighthouse here: https://dsva.slack.com/archives/CKUCFQAHJ/p1693357390891949.
  • Lighthouse's response:

    This is the actual decision date in most cases, but I'll also follow up with Caseflow to see what the range is.

• [x] Think about what it would look like to make 9 async requests for contestable_issues of every benefit_type. Specifically, how to do that on the front-end, how to handle failed requests and incomplete lists, and whether there would be any non-technical consequences from making requests for every benefit_type.
  • For context, assuming there were no consequences to doing so, getting a complete list of contestable_issues would resolve the duplicate claims problem (where Veterans add already added issues, because they don't see the issue on the AddIssues page) and avoid any content/disclaimer problems (i.e. having to explain that the Veteran is only seeing an incomplete list). According to BVA, the only consequences would be the possibility that some Veterans would "throw spaghetti at the wall to see what sticks."
  • If we decide not to make requests for every benefit_type, then we will need to understand the scope of the compensation benefit_type, including whether the compensation benefit_type` encompasses disability or non-disability claims only. According to BVA, compensation is disability-only.
  • I've asked Lighthouse for more context here: https://dsva.slack.com/archives/CKUCFQAHJ/p1693357390891949.
  • Lighthouse says (see slack thread in prev. bulletpoint) that 1) we should really migrate to their new v2 endpoint, but that 2) both endpoints return all contestable issues, for all benefit_types, so long as we don't specify a benefit_type. Which means we wouldn't need to think about what it would look like to make 9 async requests for contestable_issues. 🎉 .
• [ ] Confirm with Lighthouse that they can actually provide complete contestable_issues data for all benefit_types. For context, Bo at BVA thinks that we may not be able to get data for benefit_types like VHA and NCA.
  • I've asked Lighthouse here: https://dsva.slack.com/archives/CKUCFQAHJ/p1693357390891949.
  • Update - Lighthouse says:

    Contestable Issue data comes directly from Caseflow, so the response will contain all eligible contestable issues, there shouldn't be any missing unless there's an issue with the completeness of VA's data (which is why write-in issues are supported by the decision reviews API)

  • Based on our chat with Bo at BVA, it sounds like there are issues with the completeness of VA's data. So we should consider putting a disclaimer if one doesn't already exist.
• [x] Get more information about the v1 v. v2 contestable_issues endpoints.
  • I've asked Lighthouse here: https://dsva.slack.com/archives/CKUCFQAHJ/p1693357390891949.
  • Answered. TL;DR. Both work, but Lighthouse strongly recommends migrating to the new v2.
• [x] Ask Lighthouse whether "Debt disputes" are included within any of the benefit_types that they offer, and if not, whether they might be offered in the future.
  • I've asked Lighthouse here: https://dsva.slack.com/archives/CKUCFQAHJ/p1693357390891949.
  • Lighthouse's response:

    I'm not familiar with that benefit_type so I'll follow up with Caseflow. Since we pass through all contestable issues from them, I'd imagine they're included but I don't know under what benefit_type

• [ ] Ask Lighthouse whether pension survivors benefits is limited to that, or if it includes the full scope of "survivors benefits."
  • I've asked Lighthouse here: https://dsva.slack.com/archives/CKUCFQAHJ/p1693357390891949.
  • Lighthouse's response:

    I'll follow up with Caseflow on this too but my understanding is that it includes the full scope

• [ ] Look into whether we can use the existing "BIP" event bus directly, rather than wait for the new event bus team's MVP.
  • For context, the new event bus team is wrapping up their MVP sometime in 2024. Conveniently for us, this MVP would include events regarding decision dates.
• [ ] Reach out to VBMS and see if they would be open to sending decision date event data to a webhook we provide, or exposing a decision date query endpoint for us to consume. We can ask Emily to help us find an engineering contact there.
  • Reached out to Emily for a VBMS contact here: https://dsva.slack.com/archives/C5AGLBNRK/p1693252476948299?thread_ts=1693251475.285339&cid=C5AGLBNRK.
• [ ] Figure out what systems, besides VBMS, are responsible for setting decision dates on claims.
• [ ] Find documentation to the process for setting up InProgressForm notifications with VA Notify, figure out whether Veterans would need to "opt in" to such notifications, and if so, think about including opt-in copy on the SaveInProgress page.
• [ ] Figure out whether decision letter notification emails are sent by default, or require Veterans to opt in such emails. If the latter, then we need to think about adding opt-in content to the confirmation page.

For sprint hygiene, we'll need to think about whether to inflate the estimate on this ticket, or create separate spikes for follow-up investigations.

[saderagsdale]

@data-doge @kylesoskin I'm weeding out complexity and getting back to the purpose of the ticket. Tasks for each use case and action items below:

A Veteran wants to know the exact decision date when adding a new issue.

• Action item 1: Find the easiest way to determine whether this is available through BGS, via the bgs_ext gem.
• Action item 2: Confirm with Michael Hobson is the other benefit types are ready to use or need to be reviewed to confirm if they meet business requirements first.
• Out of scope: over-indexing on adding benefit types. If it's there, great, if not, park it.

A Veteran wants to know when their filing deadline is approaching so they can take action.

• Action item 3: Find out where filing deadline data is stored. Possibly BGS?
• Action item 4: Find out if BIP is useable as an email event bus (or another mechanism). If not. What other ways could we put deadline info in front of Veterans?

A Veteran wants to have to have their decision letter delivered via email, because decision letters don't always arrive in the mail.

• Action item 5: Find out if we can parse or otherwise surface and data from decision letters in the form flow.

I am going to look for leads on action items 1-4. @data-doge can you look into action item 5?

[saderagsdale]

Sent a note to Premal on action items 1-4.

Also reached out to VRO to find out about BIP.

[saderagsdale]

Found BIP API and Swagger documentation here, but you need to be in CAG to click the links: https://github.ec.va.gov/EPMO/bip-vetservices-claims/blob/development/bip-vetservices-claims-docs/new-integration-partners.md https://claims-test.dev8.bip.va.gov/swagger-ui.html#/

[saderagsdale]

Also got a link to this developer documentation for the VRO platform from Premal, which may have some of the data and capabilities we're looking for.

[saderagsdale]

Scheduled time to discuss action items 1-4 with Diana from VRO this Friday.

[HeatherWidmont]

@data-doge is planning on breaking this ticket down into 3 separate tickets

[saderagsdale]

@data-doge Can you tag me in this ticket when you've looked at the documentation Diana from VRO shared? I'm looking to understand how we can quickly confirm if VRO has what we need.

[data-doge]

Hey @saderagsdale sure thing. I plan on doing that today.

Also, after our conversation, and after speaking with Eileen and Ruben today, I've broken this ticket into three separate, focused spike tickets that can be worked on in parallel. The VRO-related ticket is the number one priority:

• https://github.com/department-of-veterans-affairs/va.gov-team/issues/66231 (assigned to Robin)
• https://github.com/department-of-veterans-affairs/va.gov-team/issues/66238 (assigned to me - this is the VRO-related ticket)
• https://github.com/department-of-veterans-affairs/va.gov-team/issues/66251 (assigned to Robin)

Accordingly, I'll remove this ticket from the sprint doc, and slot in these three smaller tickets instead.