Analytics Implementation or QA Support for maximum rating message in 526 form

[amylai-va]

Google Analytics Implementation or QA Request

What this form is for

More Information

Use this template to request Google Analytics implementation or QA for your product. Please fill out all brackets.

> _Please see the following information for implementation timeline:_ > New Teams: Implementation will usually be completed in 4 sprints > > - Sprint 1: Discovery/Orientation > - Sprint 2: FE Implementation > - Sprint 3: Tagging Implementation > - Sprint 4: QA > > All other teams: Implementation will usually be completed in 2 sprints > > - Sprint 1: Discovery and FE Implementation > - Sprint 2: Tagging Implementation and QA

Description

Required

• Is this request for Google Analytics implementation or QA?

  • [x] Google Analytics implementation
  • [ ] QA

• Who is this request for?

  • Employee Experience team, Benefits Claims Fast-tracking Crew (#benefits-cft-employee-exp)

• Provide the link(s) for Google Analytics implementation or QA

  • www.va.gov/disability/file-disability-claim-form-21-526ez/disabilities/rated-disabilities

• When is the launch date?

  • target: end of august

• What interactions need to be tracked?

  See comment for additional context about the requests:

  Request 1:

  • We are planning on adding copy to the rated disabilities page and releasing it to 50% of va.gov users (which we'll control via feature toggles). We'd like to track whether the user is seeing the version of the page that includes or excludes the copy. Is this something we could do in GA?

  Request 2:

  • I see that a gtm.click event fires when a user clicks on any of rated disabilities on this page
  • Would it be possible to track which disability they selected/deselected? or is this not possible due to PII concerns?
  • Here's an example from staging.va.gov:

Acceptance criteria

• [ ] All appropriate issue tagging is completed
• [ ] QA has been completed
• [ ] Events have been added to Product and Benefit Hub Content Groups

Additional information

Please refer to Platform Collaboration Cycle or the Analytics Request Touchpoint on Platform website for more information about the Collaboration Cycle.

[michelle-dooley]

@amylai-nava thank you for your request, we will review and get back to you with timing. To set expectations we have a bit of a back but are in the process of hiring a dedicated GA analyst that will be picking up all this work.

[michelle-dooley]

@amylai-nava - can you confirm that the page content is dynamic (i.e. changes per user - but same url) or is it a new url that you are only releasing to a subset of users and will eventually replace the current/old?

[amylai-va]

@michelle-dooley, good question: the page content is dynamic. the url will not be changing.

[jestutt]

Hi @amylai-nava - for this dynamic content, are you looking for an even that says it loaded for that user or are you looking for the user to have actually seen it?

[amylai-va]

Hi @jestutt, apologies for the delayed response! Can you elaborate on the difference between content is loaded vs user has seen the content?

[jestutt]

@amylai-nava
Setting the event to when the "content is loaded" is how an AB test is measured. The event captures which version of the page a user sees -- with content or without -- but, where the targeted content is below the fold, there's no way to know if the user sees that content. Your request would indicate this one.

Do you know how impressions for digital display ads are measured? A certain % of an ad must appear in the browser window for a certain minimal time for an impression to be counted based on ad council guidelines. This is similar to what I mean by "user has seen the content" only we aren't restricted by % or time -- a good thing with responsive content.

Is there a stage login with the content currently available? I'd like to check to see what if any code updates will be needed.

[amylai-va]

@jestutt, thanks for clarifying. To clarify our request, we'd like to know which version of the page a user sees -- one with or without content -- but there's a bit more nuance that might be easier to discuss on a quick call. Would you be up for that?

My assumption is that determining whether the user has actually read the content will not be straightforward or doable in GA.

The feature is not ready in staging yet but should be fairly soon.

[jestutt]

@amylai-nava I'm happy to do a quick call to discuss the nuances.

[amylai-va]

@jestutt great! are you on DSVA slack? want to message me there and we can coordinate a time?

[amylai-va]

@jestutt going to elaborate here, and then we can set up a meeting if needed.

• The feature toggle that will control what users see on the Rated Disabilities page is this one: disability_526_maximum_rating
• We're going to set it so that 50% of users see the education 100% of the time
• With the current scope of our launch, the education is one line of text that says: "You're already at the maximum rating for tinnitus". And the user group that may see the education = users who have a rated disability of Tinnitus with a current rating of 10%.

We want to know:

• Number of people who "see" our education vs. don't (so the # of people in the A & B group)
• Of those who "see" our education, the number of people who file a claim with tinnitus selected

[amylai-va]

@jestutt checking in to see if you have any questions or want to find some time to chat on a call?