[feature] Implement Analytics Tracking for Backend and Frontend

[Rperry2174]

Implement Analytics Tracking for Backend and Frontend

Now that we have separated out the cloud vs the OSS code we can more easily add things to the code in order to understand user journeys and track usage

Objective

To establish a analytics tracking system to gain insights into:

• Backend Database Usage: Monitoring database interactions, query performance, and usage patterns
• Frontend Feature Usage: Understanding user journeys, feature interactions, and engagement levels to guide UI/UX improvements and feature development

Proposed Technologies

• Backend: Prometheus
• Frontend: PostHog because it provides a detailed analytics toolkit as well as capability to record. We can also swithch it to OSS if we want (we will start with cloud)

Fullstory

fullstory is another option https://drive.google.com/file/d/1lXqM9-7rFbpKsFo4TO6nDjlbqQ5-_1l8/view?pli=1

Reach out to chris shih https://raintank-corp.slack.com/archives/C020988GK4H/p1707858775856399

https://docs.google.com/document/d/1Qle6TfzNd9jIIFMxZyKUYlpfJc0E8KIOZIUnJtGrIo8/edit

Metrics app as an example

It is using the rudderstack library and manually adding tracking code throughout. Here's an example: https://github.com/grafana/grafana-adaptive-metrics-app/blob/88875984168a094983818fa1649bad342d4171ee/src/components/PageHeader/index.tsx#L368 Here is a brief overview doc of user tracking https://docs.google.com/document/d/1mORi9tmdCTSShtQoVg3d9mMANVQCrWYGE91aSjzNcEA/edit

https://bi.grafana-ops.net/d/ba52c76b-9c45-45ad-be91-763e8438849c/adaptive-metrics?orgId=1

Requirements

1. User Segmentation: Capability to segment data by user and org for feature usage as well as backend usage
2. Dashboard and Reporting: Development or integration of dashboards for visualizing key metrics and trends

Implementation Phases

1. Research and Planning:
   • Identify specific metrics for backend and frontend tracking
   • Outline data architecture and storage requirements
2. Frontend Integration:
   • Integrate PostHog (or some other method) for frontend analytics
   • Identify key user journeys and interactions for tracking
3. Backend Integration:
   • Configure Prometheus for backend metrics collection and storage
   • Define custom metrics for database interactions and performance

Frontend

We are already using Posthog for og-pyroscope-cloud so I believe we can just continue to use this and see if it works. I'm not sure how things will work inside of a grafana environment. Eventually I hope we can switch to faro, but for now we will have to use this as it is more fully featured.

Many things "just work" out of the box so I think we can start by just adding the posthog sdk to cloud profiles page and see waht happens.

Backend

This will likely need to come from somewhere other than the app plugin, but some key metrics to track:

• Active users (daily, monthly, weekly)
• Active orgs
• Daily queries (per user / per org)

[grafakus]

I started a document to keep track of what we've tracked so far: Pyroscope App Plugin Analytics

[Rperry2174]

we should also consider after a certain amount of time with no usage of a feature should we consider to stop supporting that feature (once we have tracking enabled)

[grafakus]

@grafakus add the dashboard link here