Google Analytics user segmentation

[lund5000]

User story

As a Caseflow product manager, I want to segment our users in Google Analytics so I can better understand how they use Caseflow.

Acceptance criteria

• Product managers can create segments in Google Analytics and accurately report on data for the following user groups
  • Intake
  • Hearings
  • Judge
  • Attorney

Out of scope

• Tracking organization membership (we'd probably want this for VSOs that tend to have a 1:1 membership, but might be too complex for VA employees because GA doesn't support arrays/lists of values for custom dimensions)
• Event tracking (we want it eventually, but let's start with segments)

Background/context

Without more information on our users, the data reported in Google Analytics doesn't provide us with much insight. On Hearings, for example, I'd like to start getting a better understanding around questions like:

• How much time are hearings users spending in Queue pages (i.e. dealing with tasks) vs. Hearings pages (i.e., scheduling Vets)?
• Which visits to Hearings pages come from Hearing users vs. attorneys and judges? How does their behavior differ?
• How engaged are VSOs with Caseflow? Are there specific VSOs that are more engaged than others?

Being able to segment our users would be an important first step towards this understanding.

Technical notes

We only get 20 custom dimensions to work with in Google Analytics, so we should make sure we plan carefully.

The User-ID feature of Google Analytics seems more aimed towards sites with different unauthenticated and authenticated user experiences. You can create a view that only includes users with a User-ID, but it can't be used to create user segments nor can it be used as a dimension in custom reports. It could still be useful in getting more accurate Users counts in our reports, but doesn't add much more value than that for us.

Yoom cautioned that relying on uncached data in the User model to populate these dimensions could slow down Caseflow.

Resources/other links

• Custom dimensions and metrics documentation

Open questions

• How concerned should we be about engineers impersonating users on production to diagnose and fix issues?
• Are there other user categories I'm not thinking about that we should track as segments? For example, how are VSO users identified in Caseflow?

[yoomlam]

Notes:

• list of the valid dimension-metric pairs: Dimensions and Metrics Reference linked from Dimensions and metrics
• Custom dimensions & metrics
  • Be aware of "Limits and caveats"
  • "20 indices available for different custom dimensions and 20 indices for custom metrics in each property"
  • "Custom dimensions cannot be deleted"
  • "use them as Segments and secondary dimensions in standard reports."
  • I followed these instructions to create a custom dimension (details below)

Custom dimension: db_user_id

• index: 1
• scope: User
• Code to send data for this custom dimension:

  var dimensionValue = 'SOME_DIMENSION_VALUE';
  ga('set', 'dimension1', dimensionValue);

[yoomlam]

Regarding GA's built-in User-ID capability.

The Reporting section of this page states:

Although the session data is available via the Core Reporting API, the userId field is not available as a dimension in reports and can not be exported from Analytics.

And the Limits sections states:

The following limits are currently in place for the User ID feature:

• Cross Device reports enabled by the User-ID feature have a maximum date range of 90 days.
• The User-ID value can not be queried as a dimension in reports in either the web interface or the APIs.

This feature is currently disabled and requires us to "agree to the User-ID Policy before you can enable the feature", one of which is:

 You will give your end users proper notice about the implementations and features of Google Analytics you use (e.g. notice about what data you will collect via Google Analytics, and whether this data can be connected to other data you have about the end user). You will either get consent from your end users, or provide them with the opportunity to opt-out from the implementations and features you use.

@lund5000 Do you think you'll want to enable this feature? I'm unclear as to the added benefits compared to the db_user_id custom dimension I created (mentioned in the previous comment)?

[lund5000]

@yoomlam Cross-posting here from Slack: since we're already sending userId I think we should enable the User-ID feature to get the minor advantage of cleaner Users counts in our reports.

[yoomlam]

@lund5000 #15315 was just deployed, and I'm able to create a custom report showing pages visited by user. Let me know what you'd like to do next. I imagine you'll want to filter or group by user roles. If I provide a list of user IDs and their user groups in a spreadsheet, do you think you can combine it with GA's data to answer your questions? If not, we'll have to create another custom dimension that includes the user's user groups.

[lund5000]

@yoomlam If you can provide that I can take a look and likely build out those user segments. Thanks!

[pranaygp]

Hey. I don't mean to take away from the work you've put into setting up Google Analytics for this, but reading the original issue description and user story, I think https://windsor.io could solve your problem really well. Since you already track userId, you should quite easily be able to populate Windsor with your users' records