[DevToolbar] Analytics for which orgs have the toolbar enabled

[ryan953]

I would like to have metrics/analytics on the server side to show what endpoints the toolbar is hitting. This should be possible by looking at incoming requests to the api where the ?queryReferrer=devtoolbar url query param is set.

The purpose of these metrics/analytics would be to:

• Detect when an organization and/or project has setup the toolbar
  • "has setup the toolbar" also depends on having the project setting configured, which is another metric we should have visibility into
• Log the volume of api requests generated by people using the toolbar. it shouldn't ever generate so much load as to be an issue, but we should have visibility into it
• See which endpoints are hit most often from the toolbar, so we can infer usage of different panels
• See which endpoints ar ehit most often from the toolabr, so we can think about optimizations to better suit use-cases

With this kind of backend logging we'd be able to make a trade-off: no user-specific logging, but it's also not something users can opt-out of. We want to observe the infra and load.

So we should aim to create some charts like:

• Total number of orgs using (sending requests with the query param) the toolbar
• Total number of projects using (sending requests to a specific endpoint with the query param) the toolbar
• Total unique viewers/users of the toolbar
• unique users per org
• Volume of requests per endpoint, where the query param is set.

It would also be cool to have stats about the orgs that are using the toolbar. So like a table with columns: "org_slug, # of users total, # toolbar users, # issues, # issues resvoled" So that we can start to learn about how the toolbar can be more helpful. This might not need to be on a dashboard, more like a one-off (or something we query every month) kind of query.

[aliu39]

@ryan953 could you link the sample dashboards/charts we were looking at?

Update: nvm, I remembered they're Hydration Errors, Rage Clicks, and Replays Search in redash. If there's other good examples lmk.

[ryan953]

Lets throw up a dashboard with some of these numbers, then close this puppy afterwards.