newrelic / developer-website

Source code for the New Relic Developer Site.
https://developer.newrelic.com
Apache License 2.0
40 stars 114 forks source link

[Dashboards] Setup funnel dashboard to show the external to internal funnel for the `see data` button #1789

Closed jpvajda closed 2 years ago

jpvajda commented 3 years ago

Relates: https://github.com/newrelic/developer-website/issues/1756

Issue

We are trying to understand the install funnel from the public site to when a user clicks on the see data button in Virtuoso experience.

This funnel can be nicely visualized with the in-product Tessen events, but not yet with the public Tessen events. It seems we have pageActions for the funnel events externally and we need to probably setup tessen events for these so we can make a single funnel query as such

Example Funnel 🔽

Screen Shot 2021-10-22 at 8 22 43 AM

Funnel steps

External Site

  1. User clicks install button on a quickstart
  2. User is taken to the guided install

Internal

  1. user goes through all the steps to install
  2. user clicks on see data

Funnel that works for in the product cases.

FROM TessenAction
SELECT
funnel(email as 'user',
  WHERE eventName = 'DEVEX_New Relic IO_initialLoad' as 'Visit New Relic I/O', 
  WHERE eventName = 'DEVEX_New Relic IO_listingClicked' as 'View details', 
  WHERE eventName = 'DEVEN_Quickstart Details_clickInstall' as 'Click install', 
  WHERE eventName = 'INSTALL_QuickstartInstallationPlan_viewQuickstartInstallationUI' as 'Load install',
  WHERE eventName = 'INSTALL_QuickstartInstallationPlan_viewInstallStep' as 'View install step',
  WHERE eventName = 'INSTALL_QuickstartInstallationPlan_deployQuickstartComponents' as 'Deploy components',
  WHERE eventName = 'INSTALL_QuickstartInstallationPlan_clickSeeYourData' as 'See data'
)
WHERE email NOT LIKE '%@newrelic.com' LIMIT MAX

Current funnel for the external site cases

FROM PageAction
SELECT
funnel(session,
  WHERE actionName = 'TDEV_QuickstartClick_instantObservability' as 'View details',
  WHERE actionName = 'TDEV_QuickstartInstall_instantObservability' as 'Click install'
)
SINCE 1 week ago
LIMIT MAX

Acceptance Criteria

note: timebox to 3 points for research.

jpvajda commented 3 years ago

I don't think this is going to be possible right now, there's no unique identifying attribute that all the eventName 's use to track a user. The 2 PLG queries referenced #1756 have something called anonymousId which would probably be something we would need to add to our events.

Even with a CustomViz theres no way to link a specific users install click with a signup event. We'll need to consider adding something like that -- from @rudouglas

cc @polfliet

jpvajda commented 2 years ago

@JuliaNocera is also interested in the outcome of this

zstix commented 2 years ago

This is blocked for the same reason that the other funnel ticket is blocked (https://github.com/newrelic/developer-website/issues/1756): we need the developer site to have the latest version of the theme which includes the extra anonymousId field. More details in this comment.

zstix commented 2 years ago

We were able to correlate some information, but it's only from existing users. This is because we have a customer_user_id attribute that we can correlate across all events:

FROM TessenAction
SELECT
funnel(customer_user_id as 'user',
  WHERE eventName = 'TDEV_QuickstartInstall_instantObservability' as 'Public catalog install click',
  WHERE eventName = 'DEVEX_New Relic IO_initialLoad' as 'Visit New Relic I/O', 
  WHERE eventName = 'DEVEX_New Relic IO_listingClicked' as 'View details', 
  WHERE eventName = 'DEVEN_Quickstart Details_clickInstall' as 'Click install', 
  WHERE eventName = 'INSTALL_QuickstartInstallationPlan_viewQuickstartInstallationUI' as 'Load install',
  WHERE eventName = 'INSTALL_QuickstartInstallationPlan_viewInstallStep' as 'View install step',
  WHERE eventName = 'INSTALL_QuickstartInstallationPlan_deployQuickstartComponents' as 'Deploy components',
  WHERE eventName = 'INSTALL_QuickstartInstallationPlan_clickSeeYourData' as 'See data'
)
WHERE email NOT LIKE '%@newrelic.com' SINCE 1 week AGO LIMIT MAX

^ We've added this chart to the bottom of our [DevEn] External I/O & Quickstarts Ecosystem dashboard.

If the events in the product had an anonymousId or the events in the public catalog had a sessionId then we could get the data for all users. At the moment, we don't have a unique unifying identifier to follow a user through all of these separate events.

jpvajda commented 2 years ago

@JuliaNocera I thought I'd share our findings in making this funnel.

jpvajda commented 2 years ago

we'll do a follow up issue on the the non existing user case.