Dashboard improvement: Add new KPIs to Claims dashboard - Sprint 81 work

[jennb33]

Work remaining from prior ticket #5943

• Build ETL to support these metrics being in Claims dashboard.
• Update dashboard and validate, including dashboard documentation.

Additional work planned for this sprint and not part of original ticket:

• Written interpretation of the findings with Claims so far based on new analytics.

[jennb33]

This ticket is a duplicate of 5943 and was duplicated as work would not be complete before the end of Sprint 80. 1 story point carried over for Sprint 81

[g-opzt]

Adding in the additional 3 points that were slotted for this in this sprint as well so rounds up to 5.

[g-opzt]

Revised dashboard started and first task incorporated. Remaining tasks to be added next week. Still to be determined how timeliness should be incorporated and presented as that is a metric we have not included on dashboards until this one.

[g-opzt]

Second task (Review a claim/appeal) completed today into two data sets for the dashboard. The first was for task success on par with our current standard for task completion measurement and that dashboard page was also created. The second set was for the timeliness analysis as mentioned last week. This was more involved due to a number of data discrepancies discovered.

First we have to convert from the API steps that come through into the in-app steps that Veterans actually experience; the API steps are 1-8 and these are condensed into 5 steps in the app. combining these steps leads to some challenges with meta data discrepancies between user views of the same claim in the same step. For example, some claims viewed in the same step more than once have inconsistent dates for when the claim was originally submitted. In other cases, the same step will register more than one date of the step change. These discrepancies were relatively small compared to the overall total claims (<0.01%).

There are other instances, however, where the date the claim was viewed is earlier than the date the claim was submitted (according to our analytics). This is obviously some kind of error that will require further investigation but will have to be tabled for a follow up ticket to enable the rest of the work here to be completed this sprint.

Fortunately, there are no recorded instances of views occurring before the step change date recorded so the timeliness analysis will be able to be completed regarding how long it is taking Veterans to view claims in the app after a change in step has occurred. I do think more rigorous cleaning and investigation of this data should be done but we are at a stable enough place to move forward at this time with this round of improvements.

The last struggle has been accounting for what version users are on in the app for dashboard filtering. This has been a major limitation on other dashboards but I was able to find a path forward for this board that I think I will be able to replicate on the others so we have consistent version filtering available.

There are still 5 more tasks to develop data sets for and build into the board but the work completed on the first two tasks should make the remaining tasks significantly faster to build. I am pushing up on capacity with what was ticketed here so I will time box my remaining work on Thursday for this and deprioritize any tasks that are less essential to the upcoming Q4 work on Claims.

[g-opzt]

Blocked by BQ access

[jennb33]

Per @g-opzt on 10/16/2023, they have access to monitor the queries now that are running from Looker, and they will be following up on next steps due to the BQ access blocker issue.

[g-opzt]

Per conversation with team leadership I will continue to not run any additional queries beyond what the existing dashboard queries are pulling so that we can monitor the impact of the dashboards alone more easily. I will be OOO until Oct. 17-22 so we will revisit the status of this blocker on Monday, Oct. 23.

[g-opzt]

Access has been restored but given timeline on resuming claims work this is being returned to the backlog in favor of other planned work.

[g-opzt]

Bringing back in as part of my transition work

[g-opzt]

Dealing with some incongruities where it appears that more users succeeding in viewing a screen than attempts tracked for claim decision letters. Working to determine any pattern in these outliers to confer with FE around what might be happening.