Analytics on our Analytics - Domo Engagement

[joanneesteban]

Issue Description

How might we track how VFS teams are using our dashboards in Domo?

---

Tasks

• [x] Lauren & Chloe to determine how people are being grouped in Domo
• [x] Configure Domo groups to match the VFS roster, OR using Join to update the groups
• [x] Create charts in Domo

Acceptance Criteria

Automatically track the following:

• [x] 1. Increase avg. weekly % of VFS teams using Domo
• [x] 2. Standardized dashboard usage vs. custom dashboard usage
• [x] 3. Link clicks to documentation, broken down
• [x] 4. Recurring users
• [x] 5. % of clicks into card details (depth of use)

[chloedotbrown]

Created an ETL that generates a new datasource based on Domo's Activity Log, People, and Group reports, as well as the VFS Team Roster and an object table with everything on the Forms dashboard labeled. I'm holding off adding Content and Search, in case we end up adding/swapping any cards and we need object IDs to be stable. But this will allow us to slice up view data by VFS teams vs. other uses and Standardized dashboards vs. other Domo content.

Small note that in current data structure, it's especially important to count unique users, since POs will have rows for each of their products. If this ends up causing issues in the actual build, I'll update dataflow accordingly.

In the meantime, working spreadsheet with more notes on KPIs and datasources has been shared with all Analytics team members.

[chloedotbrown]

Adding @JasonTheBearded to this ticket for situational awareness about the ETL portion of this work.

I brought this issue up in our data chat this morning, but documenting it here for better traceability! A downside with the current dataflow is that Domo does not make it easy to append ETLs (instead of replacing full datasources each time they run). This may become an issue for us as people cycle on/off VFS teams, as that person's entire viewing history will be categorized with by their current status, not the one they held at the time. For example, if a person on a VFS team views the dashboard 20x in July (sidenote: don't I wish!), but in August they're moved off to another contract, then when we look back at our July 2021 data, it will appear as though those 20 views came from non-VFS viewers.

I'm not sure how big of an issue this will be for us – in part, because we just don't know how people will or won't use these dashboards. But it does look like one workaround for this might be to set up the ETL as a Recursive Data Flow. I haven't had time to dig into this today and will probably focus on the mockups instead tomorrow, but will pick back up and investigate whether this would be any help for us once I'm back early next week, unless anyone's able to take a peak beforehand!

[joanneesteban]

Hi! Question for y'all: we'll have data issues around folks leaving VFS or changing teams?

[joanneesteban]

Head's up, @chloedotbrown - everyone on the VSP Roster should be filtered out from our AooA. The most important folks to filter out are members of our own team, including VA.gov Platform Analytics & Insights & Participant Tester.

[chloedotbrown]

Great! I'm already pulling out activity from our team, but I'll add the Participant Tester as well. As for the rest of the Platform crew, using Domo groups with a VSP in the name seems like the easiest approach, so long as they're accurate.

@laurensimpson14 this may be a question for you! Are these groups pretty actively maintained?

[chloedotbrown]

:: Updated ::

With GCIO email addresses changing, we will need to coordinate updates in Domo and on the VFS roster for join to work properly. So far, it looks like this may be an isolated incident of 1 early adopter VFS member updating their Domo profile, but it flags the issue for what will happen when full batch is updated in either place. If these updates are staggered, GCIO VFS members will be mis-categorized as Other users in new data that gets added to our AooA source.

For now, I've updated the VFS roster for the 1 user and have set up a card to help me catch future mis-categorizations if other folks start changing their Domo profiles on an ad hoc basis. I'll also pull the historical data again this afternoon so yesterday's data for this user will be correct.

However, once we have a sense of when/where these emails will be updated en masse (Domo vs. VFS roster first), we should coordinate updates to the other part so that historical data doesn't have to be re-run.

[chloedotbrown]

Dashboard card creation and layout complete. View here.

[chloedotbrown]

Closing ticket.