jonwehausen commented 4 years ago

Google Analytics Implementation

Description

We would to refactor our current implementation of facility locator search tracking -- currently, we are tracking according to user timing, we would like to refactor this to be events
We are currently suspect to a GA limitation on user timing functionality that samples the search volume we can collect up to 1% of total pageviews -- see resource here.
Making this change will inherently give us all accurate unsampled search data and flexible reporting to tailor more specifically to their insights
Who is this request for?
- Michelle Middaugh -- recommendation for improvement by Analytics & Insights team
What team/project is this work for? Facility Locator
Provide the Domain where Google Analytics needs to be implemented i.e. www.staging.va.gov

PII includes specific address components passed in search box

Acceptance Criteria

[ ] Facility locator searches are now captured as events rather than user timing tags
[ ] QA has been completed
[ ] VFS team completes brief VSP Collaboration Cycle Feedback survey

Definition of Done

[ ] All appropriate issue tagging is completed
[ ] All AC completed

jonwehausen commented 4 years ago

Notes from Pointing Session

1) Establish a DL (containing no PII)

Jon to reach out to Michelle on scheduling Connect with developers on opportunities for pushing map data into DL

Convert existing user timing tracking to event tracking

Maintain the current dropdown and timing components we're capturing today

Increment a total count custom metrics for each search

Replace chart on Facility Locator KPI dashboard with new event taxonomy

jonwehausen commented 4 years ago

DataLayer Specification - Work in Progress

@mmiddaugh / @brangi / @mpelzsherman - Want to thank you all again for your time on monday.

This comment is a drafted dataLayer specification to track some of the various components we had discussed from Monday. This is a work in progress as we comb through and refine the structure.

On Search

We currently have an existing event fl-search-results being pushed to the dataLayer with each search.
We would like to propose the addition of the following dataLayer variables with that search event to be included to unveil and enhance our insight
Please see each variables name with its intended purpose / data to store

DataLayer Variable	Example Value	Description
`'fl-facility-type-filter'`	`'VA health'`	The facility type the user used to filter on search, `undefined` if not used
`'fl-service-type-filter'`	`'Primary Care'`	The service type the user used to filter on search, `undefined` if not used
`'fl-allowed-location'`	`true` `//false`	if the user used the allow location functionality to auto-search, `true` if so, `false` if not
`'fl-results-returned'`	`true` `//false`	`true` if the search returned ANY results, `false` if no results returned
`'fl-total-number-of-results'`	`101`	Contains the full number of results returned ,
`'fl-total-number-of-result-pages'`	`5`	Stores the total number of result pages returned for the search
`'fl-zip-code'`	`'55555'`	Contains the zip code of the search IF the user's search had a zip code, otherwise used `undefined`,
`'fl-city'`	`'seattle'`	Contains the city of the search IF the user's search had a city, otherwise `undefined`,
`'fl-state'`	`'washington'`	Contains the state of the search IF the user's search had a state, otherwise `undefined`
`'fl-country'`	`'united states of america'`	Contains the country of the search IF the user's search had a country, otherwise `undefined`
`'fl-closest-result-distance-miles'`	`32.4`	Stores the top result's distance
`'fl-time-to-return-results'`	`4023 //in milliseconds`	Stores the total time taken to return the search results We do currently have a GTM-only solution for this, however, if a relatively low lift to get within the DL, we would like to repurpose to push with this event

Clicks on Map Pin Clicks & Letter

Event / DataLayer Variable	Example Value	Description
`'event'`	`'fl-map-pin-click'`	Event push with each map pin-specific click
`'fl-facility-type'`	`'VA Health'`	Facility type of the facility clicked
`'fl-facility-classification'`	`'VAMC'`	Classification of facility clicked on
`'fl-facility-id'`	`'vha_688'`	Facility ID of facility clicked on
`'fl-facility-facility-name'`	`'Charlotte Hall VA Clinic'`	Facility name of facility clicked on
`'fl-facility-distance-from-search'`	`32.4`	Distance of facility clicked on from search params

Clicks to Zoom In/Out on Map

Event / DataLayer Variable	Example Value	Description
`'event'`	`'fl-map-zoom-in' //or 'fl-map-zoom-out'` consistently this value for each interaction type	Event push with zoom ins and zoom outs

Clicks on Search Result Facilities

Event / DataLayer Variable	Example Value	Description
`'event': 'fl-results-click'`	consistently this value	Event push on ANY search result click
`'fl-facility-name'`	`'Charlotte Hall VA Clinic'`	Store the name of the facility
`'fl-facility-type'`	`'VA Health'`	Store the type of facility
`'fl-facility-id'`	`'vha_688GD'`	Store the facility ID
`'fl-facility-classification'`	`'Primary Care CBOC'`	Store classification of facility
`'fl-result-page-number'`	`4 //if on the 4th result page`	Store the result page number user is currently on (pagination number)
`'fl-result-position'`	`5 //if the 5th result on the page`	Store the search result position in the list

Outstanding / Follow-up Questions:

@brangi / @mpelzsherman Please let us know if there is anything in each of these items that will be a more difficult lift to surface or you have any other clarifying questions with
@bsmartin-ep @bmcgrady-ep when one of you has some free time -- if you could take a look at this and let me know any adjustments / improvements you see or items I potentially missed that we should include

jonwehausen commented 4 years ago

Follow-up and Spec Adjustments

@mmiddaugh @mpelzsherman @brangi @joanneesteban Thank you again all for your time today. As a follow-up to some of the adjustments / enhancements we discussed, please see the below notes:

On Search - @brangi & @mpelzsherman

Understanding this is going to take a few different iterations of dataLayer work, and trying to cut-down on parse-out code necessary -- we can begin with the approach of taking and utilizing (what we can) of what the API gives us directly in its raw unstructured nature. We can cleanse the data on our side in accordance with PII best practices for Google Analytics.
Leveraging this documentation, the fields we can use to start with are query, place_type, and place_name for the response returned that is most relevant according to Mapbox (top result with a relevance of closest to 1).
We can ignore the original variables from above for each itemized (i.e fl-city) address piece specifically in lieu of this format below.

DataLayer Variable	API Value	Example Screenshot
`'fl-searched-query'`	`"query"`
`'fl-mapbox-returned-place-type'`	`"place_type"`
`'fl-mapbox-returned-place-name'`	`"place_name"`

On Tracking Map (Double Click or Hand Click-and-Drag) Movements

It looks like the map does indeed render a new search with even the smallest of mouse movement or double click, so in order to normalize this data and increase data quality & meaning -- We could try to set a threshold value the map must be moved from point A to point B (i.e previous coordinate or searched location to newly moved coordinate) before we write the code to record the dataLayer 'event'. @mmiddaugh we would of course leave it up to you for what that threshold should be and priority for writing the logic to do so. Perhaps a miles threshold?
It may be helpful to capture at time of map move search completion the user's zoom depth at this point as well to understand proximity, view and overall scale of what the user is actually seeing on the map as well -- that could give us a better sense at time of insight of it perhaps made sense for that user to use the map based on how much they were zoomed in. This also gives us a better way to understand mobile users who naturally have smaller real estate to work with and zoom more often. Interested to hear @brangi and @mpelzsherman your feedback on this, open to your suggestion.
For each map movement of significant threshold, we can push the event fl-map-movement-search, along with the following variables:

DataLayer Variable	Example Value
`'fl-map-miles-moved'`	`'300 miles` `//output of point A to point B calculation` open for discussion based on what's available, whether qualitative or quantitative
`'fl-map-zoom-depth'`	`'<current_zoom_depth>'` also open for discussion based on what's available, whether qualitative or quantitative

Follow-up Items to consider - @mmiddaugh

We can add/remove variables that are unneccessary or don't feel like they provide additional value and help reduce scope / testing etc.
Certainly open to your teams priorities and scheduling for how we want to structure the timeline:
- Perhaps we try to do the search results returned spec and the search result clicks during 1 sprint and the map movements and zoom interactions during another. What are your thoughts there? Happy to break-out into separate issues as well.
Last Question: Could you all provide for us any further documentation you all may have on your planned upcoming auto-complete type search functionality?

jonwehausen commented 4 years ago

ccing @bsmartin-ep and @bmcgrady-ep for visibility.

mmiddaugh commented 4 years ago

@jonwehausen @brangi @mpelzsherman for what it's worth, I spoke with Dave about the 'fl-facility-classification' and there may be value to this data in the future.

I moved the datalayer element items into an issue on our board which is now tagged with our "needs-grooming" label so we can start thinking about how to break this apart and prioritize.

mmiddaugh commented 4 years ago

@jonwehausen here is more info on the Predictive GeoLocation discovery. Please note we haven't defined our next steps or timeline yet.

jonwehausen commented 4 years ago

Thank you for this @mmiddaugh we will take a look at this!

Also appreciate you asking about the classification data.

We are free to work at your teams pace & priorities from this point forward.

jonwehausen commented 4 years ago

Next Steps

@bmcgrady-ep to perform QA + documentation of dataLayer
@bsmartin-ep to stage GTM implementation of dataLayer specification + determine areas where hit volume should be mitigated prior to tag testing
- Also look at deprecating custom timer and user timing tag in place of events

jonwehausen commented 4 years ago

QA

ccing you both here: @brangi @mpelzsherman

On Search Click/Enter

Description of Interaction	Test Status	Notes
Search with VA health as facility type and primary care as service type	✔️ PASS	N/A
Search with just community pharmacies (service type disabled)	✔️ PASS	N/A
Search with Vet Centers facility type	✔️ PASS	N/A

On Search Results Returned

Description of Interaction	Data Layer Screenshot	Test Status	Notes
Search results returned		❌ DOES NOT PASS	• Similar to how the correction on the `fl-map-pin-click` event causing multiple messages, each of these separate messages containing Data layer variables should all be sent with the `fl-search-results` event • Once we get this corrected we can take a more thorough look at the Data Layer as in the examples below

On Search Results Click

Description of Interaction	Data Layer Screenshot	Test Status	Notes
Click on first search result in results list		✔️ PASS	N/A
Click on third result on first page of search results list		✔️ PASS	N/A

Map Pin Click

Description of Interaction	Data Layer Screenshot	Test Status	Notes
Click on 'E' in map		✔️ PASS	N/A

Map Zoom In/Out

Per @mpelzsherman comment here, perhaps a better approach to help conserve GA hit volume is to include both 'fl-map-miles-moved' and something like 'fl-current-zoom-depth' along with the event fl-search to conserve traffic volume

Description of Interaction	Data Layer Screenshot	Test Status	Notes
Zoom In		❌ DOES NOT PASS	• See comment above in this section, consider pivot
Zoom out		❌ DOES NOT PASS	Same issue as described prior

brangi commented 4 years ago

@jonwehausen thank you for this feedback. I have one request for you:

On Search Results Returned

Description of Interaction Data Layer Screenshot Test Status Notes Search results returned ❌ DOES NOT PASS • Similar to how the correction on the fl-map-pin-click event causing multiple messages, each of these separate messages containing Data layer variables should all be sent with the fl-search-results event

• Once we get this corrected we can take a more thorough look at the Data Layer as in the examples below

Would it be possible for you to specify with a more detailed list of which kind of messages you are referring to messages containing Data layer variables should all be sent ?

jonwehausen commented 4 years ago

Hi @brangi ,

Here are the full data layer samples of each event..each event should be pushed simultaneously along with their respective variables. All of the variables you see below should be pushed along with the same recordEvent() for each interaction.

From what we could see, the variables were populating with the correct values, but they were all being pushed as separate messages. When these variables are pushed together, it allows us to reference the metadata in accordance with each event to implement tracking.

When ALL search results are returned

'event': 'fl-search-results',
'fl-facility-type-filter':
'fl-service-type-filter':
'fl-allowed-location':
'fl-results-returned': 
'fl-total-number-of-results':
'fl-total-number-of-result-pages':
'fl-searched-query':
'fl-mapbox-returned-place-type':
'fl-mapbox-returned-place-name':
'fl-closest-result-distance-miles':
'fl-time-to-return-results':

On Map Pin Clicks

'event': 'fl-map-pin-click'
'fl-facility-type'
'fl-facility-classification'
'fl-facility-id' | 'vha_688'
'fl-facility-facility-name'
'fl-facility-distance-from-search'

On Search results clicks

'event': 'fl-results-click' 
'fl-facility-name' 
'fl-facility-type'
'fl-facility-id'
'fl-facility-classification'
'fl-result-page-number'
'fl-result-position'

brangi commented 4 years ago

thanks, @jonwehausen, I think it's going to be challenging to gather all those messages for event': 'fl-search-results', at the same time, because of the way our API calls are designed (for vetsAPI and then for MapBox). But let me see what I can do.

jonwehausen commented 4 years ago

Okay @brangi, thanks for this -- We want to try to conserve our hits but provide as much metadata as we can where we can, keep us posted on the complexity here and if we need to pivot to an alternate solution. Thanks again

jonwehausen commented 4 years ago

This is now ready for thorough QA cc: @bsmartin-ep @bsmartin-ep

bsmartin-ep commented 4 years ago

@jonwehausen -

QA of this workspace

Interaction	Data Layer	Tag	Notes
Search
Search Results			Missing `fl-service-type-filter` and `fl-allowed-location`
Pin Click			Looks like we're missing `fl-facility-id`. Remove that?
Zoom Out
Zoom In
Results Click
Map Move

@jonwehausen - would it be possible for the developer to clear the fl-map-miles-moved dataLayer variable after it is set? Otherwise, I'm having trouble differentiating between the regular search and the map move from the dataLayer alone.

window.dataLayer.push({'fl-map-miles-moved': undefined});

jonwehausen commented 4 years ago

@brangi Please see @bsmartin-ep comment ⬆️

jonwehausen commented 4 years ago

Thanks for doing the QA here, @bsmartin-ep really appreciate it...A couple things here:

1) It would be great if we can have at least a couple days of live data flow into staging to get a taste of volume / data quality before we go live (will probably help identify where we can clean things up further too) 2) This should help also give us an indicator on what type (if any) throttling we need to do

mmiddaugh commented 4 years ago

@jonwehausen for your visibility:
Capturing adjustment here The issue will be brought into our new sprint which starts tomorrow (10/7)

jonwehausen commented 4 years ago

@mmiddaugh Thank you very much! We appreciate it.

@bsmartin-ep #14453 will include the adjustment you requested -- let's keep an eye on our staging data as this change rolls in.

jonwehausen commented 4 years ago

cc: @bsmartin-ep

This has been published to production as of 10/12 2:20pm EST.

Let's continue to monitor production utilization.

jonwehausen commented 4 years ago

@bsmartin-ep @mmiddaugh closing this implementation ticket per demo last week

Please find ticket to setup the BigQuery view at #15000

cc: @joanneesteban for visibility

department-of-veterans-affairs / va.gov-team

Revamp Implementation of Facility Locator Search Tracking - Convert to Events #10987

Google Analytics Implementation

Description

Acceptance Criteria

Definition of Done

Notes from Pointing Session

DataLayer Specification - Work in Progress

On Search

Clicks on Map Pin Clicks & Letter

Clicks to Zoom In/Out on Map

Clicks on Search Result Facilities

Follow-up and Spec Adjustments

On Search - @brangi & @mpelzsherman

On Tracking Map (Double Click or Hand Click-and-Drag) Movements

Follow-up Items to consider - @mmiddaugh

Next Steps

QA

On Search Click/Enter

On Search Results Returned

On Search Results Click

Map Pin Click

Map Zoom In/Out

On Search Results Returned

When ALL search results are returned

On Map Pin Clicks

On Search results clicks

QA of this workspace