PCIU Discovery into the User Experience when contact information is missing

sethdarragile6 commented 7 months ago

Issue Description

In order to make a more informed decision about how to proceed with the PCIU migration/removal, we need to better understand what the user experience is when contact information is not returned from VA Profile (mostly address info). Common sense predicts that there will simply be no information to prefill, and the user must enter it- annoying, but not impeding the filing with errors, quirks, etc. The truth might be different.

Factors to check:

sometimes there is a message informing the user that their info will prefill. need to understand the conditions around that message to manage their expectations
check missing phone and email scenarios
check outage scenarios (vs. simply returning no data)

Tasks

[ ] see above

Acceptance Criteria

[ ] Determine what the user experience is for missing address, phone and email information returned from VA Profile (without PCIU)

How to configure this issue
[ ] Attached to a Milestone (when will this be completed?)
[ ] Attached to an Epic (what body of work is this a part of?)
[ ] Labeled with Team (product support, analytics-insights, operations, service-design, Console-Services, tools-fe)
[ ] Labeled with Practice Area (backend, frontend, devops, design, research, product, ia, qa, analytics, contact center, research, accessibility, content)
[ ] Labeled with Type (bug, request, discovery, documentation, etc.)

sethdarragile6 commented 7 months ago

Here is the note when starting an application, FYI. While well-meaning, it might raise expectations when pre-fill fails. Side note: can this app be filled out without being logged in? Should add this to the testing matrix here, at the very least. In this case, there is zero chance prefill should work.

sethdarragile6 commented 7 months ago

I see no option to start this app without logging in, so never mind the above. Still, it could be considered to remove mention of prefill in order to lower expectations. 😂🤔🤷

sethdarragile6 commented 7 months ago

Locally, with mocked authentication (user 1 id.me), PCIUAddress is inaccessible. An error response comes back that it is unavailable, but I am able to fill in the form and go all the way to submission. 👍

sethdarragile6 commented 7 months ago

Testing this organically is challenging, if not impossible (well, not feasible, at least). Almost all staging accounts (if they work) seem to return prefill contact info (and how organic is THAT, even, in a world of mocked or otherwise staged data. I cannot login with my own account on Staging or Production, as I am missing EDIPI and BIRLS ids. All 3 canary testers that i met with for the initial release had successful prefills. So, as a relatively low lift (compared to things like traffic interception tools, etc), I decided to use DataDog.

Similar to the way I initially investigated missing addresses, I did the following for 4-5 different missing address instances from the DataDog dashboard:

On the first graph (for Flag OFF), I deselected all metrics except for "mailing_address missing"
Clicked on a purple bar and chose View related logs
Selected one of the log entries and copied the client_ip
In an new tab, I opened a Log Explorer window and filtered for @client_ip:xxx.xxx.xxx.xxx (wrap the IP with quotes for IPv6 addresses)

In this view, we're able to see the progression of that user's "journey" through the app toward submission. I used this on 4-5 different missing address occurrences that happened within the last week (interestingly, there seem to be relatively more of these lately- though there does seem to be an increase in prefill requests generally over the past couple months). With these instances, I was able to observe that navigation through the app continued unabated, visiting recognized referring pages such as terminally-ill and new-disabilities/add. With some cases, they went as far as successfully submitting the application. If there had been an error or disruption due to the missing addresses, we would expect to see much longer time gaps between requests after the prefill point- similar to the cases I've observed of ITF failures where they were stopped cold, or came back hours later to try again.

All of this, combined with the fact that locally there is no disruption (see previous comments), I feel much more confident that the user experience is inconvenient, but otherwise not disruptive.

cc @emilytheis @RakshindaAslam @alatempa24

sethdarragile6 commented 7 months ago

On another note, it looks like ~Auth Experience~ Platform might be planning to do this this sprint anyways?? 😂 Unless I'm misunderstanding... https://app.zenhub.com/workspaces/disability-experience-63dbdb0a401c4400119d3a44/issues/gh/department-of-veterans-affairs/va.gov-team/76171

RakshindaAslam commented 7 months ago

Thanks @sethdarragile6! @emilytheis - this suggests that we should be able to progress with the launch at this time. Let us know if you have any concerns with that.

emilytheis commented 7 months ago

Making this decision based on whether or not we have a sense of if VA Profile is going to fix the root cause of the latency. It's more of a timing question than anything :

Link to question https://dsva.slack.com/archives/C7TE0PFTL/p1710528125615529?thread_ts=1708732310.734809&cid=C7TE0PFTL

department-of-veterans-affairs / va.gov-team