department-of-veterans-affairs / va.gov-team

Public resources for building on and in support of VA.gov. Visit complete Knowledge Hub:
https://depo-platform-documentation.scrollhelp.site/index.html
283 stars 204 forks source link

[Epic] Use Analytics to understand user journey through 1010 forms #56764

Closed alexseelig closed 10 months ago

alexseelig commented 1 year ago

As a team member, I would like to understand how users interact with the 1010 forms so that I can understand their pain points, their successes, and ensure that as many Veterans have easy access to care as possible.

One issue that was identified: A small percent of monthly applicants were having issues with the start application buttons within the 1010ez form

AS Note: leaving Epic Outline down below in case we need to turn this into an Epic

Product Outline

Link

High Level User Story/ies

As a Data analyst, I need to research & vet the feedback given by veterans trying to apply for medical benefits; so I can suggest areas of improvement for the broader team to address and facilitate a better user experience within the 1010 forms.

Background Context

Per the DOMO KPI dashboards for the 1010ez form; roughly 84% of users do not make it past the first 7 steps of the application. There are many reasons why a veteran would not go past the first couple of step of the form outside bugs and errors. However, there is a wealth of feedback data from veterans that has not been utilized, we wanted to understand what were veterans pain points on the up front of the form.

Hypothesis or Bet

If We identify the hurdles or problem areas of users trying to apply for medical benefits Then We can expect that applicant drop off rate will decrease on the upfront of the 1010ez form.

OKR

A measurable decrease in applicant drop off rate within the first 7 steps of the 1010ez application funnel.

Definition of done

When we can spin up a few stories for the boarder team to bring to stakeholders' attention and address those issue, if any.

What must be true in order for you to consider this epic complete?

N/A

Take into consideration Accessibility/QA needs as well as Product, Technical, and Design requirements.

How to configure this issue

luis-simauchi commented 1 year ago

DATA DOG FINDINGS: here is a typical example of how a chain of errors can occur on page load before the user gets the chance to press a start application button.

Capture 2 Capture 3
luis-simauchi commented 1 year ago
Session ID | Error | When the Error loads -- | -- | -- 49baaa6a-a8a4-487d-917d-17841227bf19 | Error: Unexpected quantifier  at undefined @ https://www.va.gov/generated/node_modules_department-of-veterans-affairs_web-components_dist_esm-es5_va-modal_2_entry_js.entry.js:1:13971 | When page loads 49baaa6a-a8a4-487d-917d-17841227bf19 | ChunkLoadError: Loading chunk node_modules_department-of-veterans-affairs_web-components_dist_esm-es5_va-modal_2_entry_js failed.(missing: https://www.va.gov/generated/node_modules_department-of-veterans-affairs_web-components_dist_esm-es5_va-modal_2_entry_js.entry.js)ChunkLoadError: Loading chunk node_modules_department-of-veterans-affairs_web-components_dist_esm-es5_va-modal_2_entry_js failed.(missing: https://www.va.gov/generated/node_modules_department-of-veterans-affairs_web-components_dist_esm-es5_va-modal_2_entry_js.entry.js) | When page loads 49baaa6a-a8a4-487d-917d-17841227bf19 | TypeError: Object doesn't support property or method 'getElementById'  at t.prototype.updateInfoMaxHeight @ https://www.va.gov/generated/node_modules_department-of-veterans-affairs_web-components_dist_esm-es5_va-additional-info_entry_js.entry.js:1:1123  at t.prototype.componentDidLoad @ https://www.va.gov/generated/node_modules_department-of-veterans-affairs_web-components_dist_esm-es5_va-additional-info_entry_js.entry.js:1:1296  at re @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:188558  at ee @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:188127  at l @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:187715  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:187477  at l @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:178031  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:177332  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:177041  at B @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:43354  at l @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:176853  at Z @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:187392  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:187354  at oe @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:188607 at J @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:187155  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:187052  at De @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:195615  at Me @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:195687  at r @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1500521 | When page loads 49baaa6a-a8a4-487d-917d-17841227bf19 | TypeError: Unable to get property 'isProxied' of undefined or null reference  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:190220  at l @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:178031  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:177332  at i @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/web-components.entry.js:2:176913  at Anonymous function @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:42106  at r @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/vendor.entry.js:2:100082 | When page loads 49baaa6a-a8a4-487d-917d-17841227bf19 | TypeError: Unable to get property 'call' of undefined or null reference  at he @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1230889  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1231007  at Anonymous function @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:13107  at some @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:24456  at he @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1230889  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1231007  at Anonymous function @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:13107  at some @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:24456  at he @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1230889  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1231007  at Anonymous function @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:13107  at some @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:24456  at he @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1230889  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1231007  at Anonymous function @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:13107  at some @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:24456  at he @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1230889  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1231007  at Anonymous function @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:13107  at some @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:24456  at he @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1230889  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1231007  at Anonymous function @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:13107  at some @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:24456  at he @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1230889  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1231007  at Anonymous function @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:13107  at some @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:24456  at he @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1230889  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1231007  at Anonymous function @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:13107  at some @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:24456  at he @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1230889  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1231007  at Anonymous function @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:13107  at some @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:24456  at he @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1230889  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1231007  at Anonymous function @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:13107  at some @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:24456  at he @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1230889  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1231007  at Anonymous function @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:13107  at some @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:24456  at he @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1230889  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1231007  at Anonymous function @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:13107  at some @ https://www.va.gov/generated/polyfills-core-js.entry.js:1:24456  at he @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1230889  at Anonymous function @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js:2:1233636 | After click on start 49baaa6a-a8a4-487d-917d-17841227bf19 | Error: Script error.  at undefined @ https://prod-va-gov-assets.s3-us-gov-west-1.amazonaws.com/generated/hca.entry.js | After click on start
luis-simauchi commented 1 year ago

Listed above is a detailed view of the errors flagged when the experience of the start of the application is not working appropriately.

luis-simauchi commented 1 year ago

list of browsers that were used during these error actions during the last 30mins

-- List of browsers -- Edge 111.0.1661.62 Facebook 406.0.0 Google 256.2.518341946 Mobile Safari 16.3 Safari 16.3

luis-simauchi commented 1 year ago

Chapley suggested that based off the order / type errors fired off, He is betting that some html is supposed to be loaded during that load process, it isn't getting loaded, and the returned value is likely a blank string.... so the getElementById is naturally going to die as strings don't have that method... there's likely an ajax call that is dying there.