Growth Metric

[adamziel]

Let's track Playground growth and share the stats publicly. Version 1 would involve measuring the number of started Playground sessions in any context (browser, node, desktop app etc.)

Done is

• There's a defined a way of measuring new sessions (that doesn't involve any PII)
• It's clear what does it cover and what are the shortcomings
• There's a plan of action to address the shortcomings
• There's a workflow in place to share updates – either manually every month, or automatically in a continuous way

Technical considerations

This requires either:

• A wasm-level call to capture the fact a new Playground was initialized. There's no other place that would assure reliable logging from custom integrations, or
• A decision not to track sessions started outside of the official Playground projects.
• Fixing the WordPress core telemetry call to wp.org. It doesn't work now. The wp_install / wp_blog headers are not set, due to the CORS issues. https://github.com/WordPress/wordpress-playground/pull/1009 https://make.wordpress.org/systems/2024/02/07/cors-requests-for-api-wordpress-org/. After fixing the above, we'd have to ensure the w.org stats looks at those fields + the Playground PHP extension.
  • Some examples of how the API requests would look in the wild would be helpful to determine how the stats could be worked, including once-off and ongoing-use.
  • What would wp-now or Studio API requests look? What’s the unique identifier on each request? etc.
  • Complication: w.org API excludes temporary/not-routable/development sites from the stats to avoid spam.

[bgrgicak]

A decision not to track sessions started outside of the official Playground projects.

I prefer this option. We need to allow opt-out of telemetry/metrics data for sessions that are started outside of our project. The easiest solution is not to track.

Our data doesn't need to be perfect, it should cover all standard usage behaviors to help us spot trends.

[bgrgicak]

Should we start sending metrics to the new Playground server?

[adamziel]

@bgrgicak sending them to the new server implies we'll need to figure out wrangling and dashboarding the data. I'd prefer Google Analytics + "share this dashboard" for now, unless there's another canonical approach in WP.org.

[bgrgicak]

Sorry, I haven't explained my thoughts completely.

The current GA implementation is limited to the browser, but if we sent all analytics data to the Playground server we would get support for other Playground apps like CLI and remote.

In that case, the Playground server could be the starting point and data could be sent to another service. Today this is GA, but it could be anything else in the future. (GA seems to support sending events over HTTP.)