Telemetry exists to better understand user signin behavior

[Daniel-Warner-X]

What telemetry do we have? What do we need? What is the level of effort for each telemetry item? https://docs.google.com/presentation/d/14_kyn6zMkrlMoZdRU4Dm6UNSC7S9wk3Yfc2hBNhe4S0/edit#slide=id.g200d5fd0836_0_1

[ahtrotta]

For reference, here is the current telemetry spec (it's up to date AFAIK).

Existing Telemetry Events:

• Open OpenAPI Instructions page: appland.appmap/view:open with Properties['appmap.view.id'] == openapi
• Open Runtime Analysis Instructions page: appland.appmap/view:open with Properties['appmap.view.id'] == investigate-findings
• Click Sign Up Runtime Analysis CTA: appland.appmap/analysis:cta_interaction

Proposed Telemetry Events:

• Navigate to a different AppMap: Not sure what this means
• Click Learn More Runtime Analysis CTA: Effort: Low, but what is the value?
• Navigate to a Code Object from tree view: Effort: Low
• Use Search in AppMap sidebar: Effort: Low
• Click on list item in AppMap sidebar: Effort: Low
• Scroll in AppMap sidebar: Effort: Unknown (might create way too many telemetry events)
• Click on Dependency Map: Effort: Low
• Click on Trace View: Effort: Low
• Click on the Filter button in the header: Effort: Low
• Click on the Refresh button in the header: Effort: Low
• Scroll in the AppMap (Dependency or Trace): Effort: Unknown (might create way too many telemetry events)
• Zoom in the AppMap (Dependency or Trace): Effort: Unknown (might create way too many telemetry events)

[Daniel-Warner-X]

This looks great @ahtrotta . Thanks. To answer your questions:

Navigate to a different AppMap: Not sure what this means -- It means did the user open one AppMap and stay there or did they open a second AppMap? Maybe a more specific measurement would be 'how long did the user spend looking at an AppMap before they closed it or navigated away? How many individual AppMaps did the user open?

Click Learn More Runtime Analysis CTA: Effort: Low, but what is the value? -- The value is learning weather or not we are presenting valuable options to the user at this critical step in the user flow. Are users availing themselves of 'Learn more'? Are users typically clicking this 'learn more' button before or after they get their first AppMap? Slides 2-3 of this presentation go into this a little deeper. Ultimately a heat map of what users are clicking on out of the 20 or so options we present would help us make decisions based on what we can change or improve and what we should leave alone.

If you had to estimate what 'Low' effort would mean in terms of hours and days, what do you think it would be?

[ahtrotta]

Okay, thanks for the clarifications!

• Navigate to a different AppMap: Based on your explanation, we basically have this covered with the appland.appmap/appmap:open event. It won't answer the question of 'how long did the user spend looking at an AppMap before they closed it or navigated away', but it would answer if and when the user opened a second (or third, or fourth, etc) AppMap.

• Click Learn More Runtime Analysis CTA: Okay, makes sense. I can definitely make this happen.

• Estimating 'Low' Effort: I would guess an hour or two for each 'Low' effort item. So to do all of the above items would probably take two or three days or so.

[Daniel-Warner-X]

@ahtrotta one more important measurement is how many people actually click on the AppLand Icon in the sidebar after install. I believe this is something we already have an event for, but we need to add it to the list.

[appland-release]

:tada: This issue has been resolved in version 0.65.0 :tada:

The release is available on:

• v0.65.0
• GitHub release

Your semantic-release bot :package::rocket: