search

department-of-veterans-affairs / va-mobile-app

"If VA were a company, it would have a flagship mobile app."
https://department-of-veterans-affairs.github.io/va-mobile-app/
17 stars 2 forks source link

Define Analytics: Availability Framework #6978

Closed StacyB2023 closed 1 year ago

StacyB2023 commented 1 year ago

Description

Today, when critical errors affect the Mobile Application, we currently do not have an immediate way to address them outside of the normal app store release and review process. As a result, users experience a broken app until it can be fixed.

For this User Story, we would like to define analytics for Availability Framework (Direct Measurements) as well as advice on any Indirect Qualitative User Experience measurements that can be done.

Epic and Use Cases for Reference HERE

Acceptance Criteria

- [x] Define Direct Measurement for Analytics including any new events that need to be added - Is the veteran seeing the critical message in the app? - Is the veteran performing the update (via the button) for the app store fix? - [x] Advice on User Experience Qual measurements (such as user testing/research sessions)
StacyB2023 commented 1 year ago

@g-opzt As we discussed for the direct measurement.

g-opzt commented 1 year ago

Ticket #6983 written for analytics for availability framework to measure direct function and impact on updates to resolve. All other measurements discussed thus far will rely on these in combination with existing analytics. The final metrics plan will be posted to Github pending @StacyB2023 discussions with @lstraghalis for research insights.

g-opzt commented 1 year ago

Blocked until I hear from Product/Research on user experience qualitative measures

lstraghalis commented 1 year ago

We don’t need to move forward with research ahead of launch unless our confidence levels were low. Spoke to Misty and she is confident in the error messaging as it stands. Going forward, we can utilize lagging indicators vis-a-vis monthly app store review analysis and reddit boards to monitor for potential issues around Veteran comprehension of the errors after launch. If there's a clear indication that Veterans are confused, then we can reassess the need for a future research study.

g-opzt commented 1 year ago

Based on the above from Liz, moving ahead with finalizing the notes on how the measurement will be done before closing out this ticket.

g-opzt commented 1 year ago

Measurement plan available in the feature documentation here.