[spike] DR | Initialize & use DataDog RUM on pages outside our forms

Mottie commented 1 month ago

Value Statement

As a Benefits Decision Reviews team member We want to see DataDog Real User Monitoring on pages outside our forms So that analyze and determine the flow leading into and out of our form

Background Context

Currently, our DataDog Real User Monitoring is initialized after the user starts the form. You won't see pages external to the form, which includes the introduction and confirmation pages.

We would also want to include this list of static pages directly related to our decision review forms:

Benefit hub detail pages:

R&S pages in the decision reviews category:

Also note this Slack discussion with Steve Albers

Tasks

[ ] Investigate how to add and initialize DataDog RUM sessions on the aforementioned static pages
[ ] Determine the approximate number of page visits, so we can determine a cost estimate of these RUM sessions

Acceptance Criteria

[ ] A clear path to adding DataDog RUM sessions to static pages has been determined
[ ] Create a development ticket for the work

Definition of Ready

[ ] Clear value description
[ ] Testable acceptance criteria
[ ] Accessibility added to acceptance criteria
[ ] Approved designs attached
[ ] Sample data provided where appropriate
[ ] Estimated to fit within the sprint
[ ] Dependencies and blockers linked

Definition of Done

[ ] Meets acceptance criteria
[ ] Code reviewed (internal)
[ ] Submitted to staging
[ ] Reviewed and approved by product and/or design

Mottie commented 1 month ago

Current progress:

I've collected a list of entity IDs and current template for each listed path:

EntityId	Template	Path	Page views per month
3071	landing_page.drupal.liquid	decision-reviews	~70700
3011	page.drupal.liquid	decision-reviews/supplemental-claim	~5460
3003	page.drupal.liquid	decision-reviews/higher-level-review	~20300
3029	page.drupal.liquid	decision-reviews/board-appeal	~8500
3010	page.drupal.liquid	decision-reviews/board-appeal/veterans-law-judge-hearing	~1300
3013	page.drupal.liquid	decision-reviews/board-appeal/after-board-appeal-decision	~760
3008	page.drupal.liquid	decision-reviews/after-you-request-review	~1800
3030	page.drupal.liquid	decision-reviews/contested-claims	~5000
3025	page.drupal.liquid	decision-reviews/insurance-claims	~480
3016	page.drupal.liquid	decision-reviews/fiduciary-claims	~870
3102	page.drupal.liquid	decision-reviews/legacy-appeals	~2560
3103	page.drupal.liquid	decision-reviews/legacy-appeals/priority-review	~2380
57215	page.drupal.liquid	decision-reviews/clinical-appeals	~2180
57308	page.drupal.liquid	decision-reviews/family-caregiver-program-reviews	~850
62667	page.drupal.liquid	get-help-with-review-request	2
23290	support_resources_detail_page.drupal.liquid	resources/choosing-a-decision-review-option	~20500
50010	support_resources_detail_page.drupal.liquid	resources/decision-reviews-faqs	~3600
64063	support_resources_detail_page.drupal.liquid	resources/hearing-coordinators-for-the-board-of-veterans-appeals	~80
8520	step_by_step.drupal.liquid	resources/how-to-check-your-va-claim-appeal-or-decision-review-status-online	~19050
53778	support_resources_detail_page.drupal.liquid	resources/requesting-a-virtual-hearing-for-a-board-appeal	~460
49994	support_resources_detail_page.drupal.liquid	resources/vas-duty-to-assist	~3270
9626	support_resources_detail_page.drupal.liquid	resources/what-your-decision-review-or-appeal-status-means	~7030

From my current understanding:

For each of the above unique templates, we need to build a new template that can include the original.
At the bottom of each new template, add a script tag that loads in and initializes DataDog.
We'll need Steve Albers to provide the cleint token and application ID for these pages
Once these pages are built, we'll need to get access to CMS, or have someone from CMS to go to each page (path or entityID?) and update the template.

Open questions:

I don't know how DataDog will behave if it is initialized & possible re-initialized on each of these pages separately.
How do we use a feature flag for this work, if needed?

Mottie commented 1 month ago

After chatting with @dsasser this is our plan of action:

I will get @saderagsdale to contact the Fran Cross to get this work added to that team's sprint work.
I will get the DataDog initialization code from Steve Albers, with sample rate percentages
I will then open a PR in the content-build repo to:
- Create an includes file with the DataDog initialization code, and
- Update these 4 liquid templates to include a conditional block which "includes" the DataDog file: page.drupal.liquid, landing_page.drupal.liquid, step_by_step.drupal.liquid, and support_resources_detail_page.drupal.liquid
Once that PR is open, let the #sitewide-public-websites team know. And when they have time, they will start their CMS work to add a flag to each template.
Some additional work will get done to add a CMS-side feature toggle (not related to vets-api feature toggles)
After that, someone from CMS will update each entity ID page to use this flag
And I will likely need to update the content-build PR to match the flag & feature toggle

Mottie commented 1 month ago

This work is blocked until we can do the following - after speaking with Steve Albers on Friday, we need to:

Collect the page view analytics and share them with him on the page with the most traffic. A ballpark number would be sufficient. This will be included in his report and determine the initial RUM session rate value (percentage of sessions to be recorded)
Discuss masking the search input on the support resource pages. Steve mentioned that some users include searches for PII & PHI unintentionally; but we should ask the search team to make sure.

saderagsdale commented 1 month ago

Steve needs an estimate of the busiest page to determine the percentage of sessions to record. He'll then write a report and give us the initialization code.

Mottie commented 1 month ago

@saderagsdale I updated the previous comment with the analytics I found - it looks like the decision-reviews page is the clear winner with about 70,700 views/month

Mottie commented 1 month ago

I reached out to Steve Albers today. He'll be in Denver at offsite next week, but will try to get work on this finished tomorrow.

Mottie commented 4 weeks ago

On hold until we determine who requested and will monitor these RUM sessions

va-albers commented 1 week ago

RUM code block & configuration has been set up & script was sent to @Mottie

jilladams commented 1 week ago

Folks on this ticket reached out to Public Websites to request a CMS feature toggle to enable this work, tracked here: https://github.com/department-of-veterans-affairs/va.gov-cms/issues/18409. We think we can accommodate that in our current sprint, but will help if y'all can provide info on hard time requirements, on that ticket. I've commented over there and tagged @Mottie

department-of-veterans-affairs / va.gov-team