[Experiment][iOS] No specific telemetry events are displayed in Glean while being enrolled in the “Social Proof on iOS "Default Browser" onboarding screen” experiment

[alicevarro]

[Notes]:

• This issue is reproducible with the following specific telemetry events as well, not just the one given as an example in the Steps to reproduce section:
  • default_browser_onboarding.dismiss_pressed;
  • default_browser_onboarding.go_to_settings_pressed.

[Affected versions]:

• Firefox Release 121.0 (36341 and 36255 builds downloaded from TestFlight).
• Firefox Release 120.1 (36298 build downloaded from TestFlight).
• Firefox Release 121.0 (Custom Build made with xCode - branch 23869f9 built on 11/27/2023).

[Affected Platforms]:

• iOS 17.1 - iPhone SE
• iOS 16.2 - iPhone 12 Pro
• iOS 17.0 - iPhone 15 (simulated device)

[Prerequisites]:

• Have a custom Firefox build containing the experiment installed on the device.

[Steps to reproduce]:

1. Open the Firefox app using a telemetry command, e,g: firefox://glean?logPings=true&sendPing=metrics&debugViewTag= name
2. Tap the “Set as Default Browser” button from the Onboarding screen.
3. Tap the Go to my Settings button from the menu.
4. Choose Firefox as the default browser.
5. Put the app in background and using a 3rd party app open any link.
6. Navigate to https://debug-ping-preview.firebaseapp.com/ and observe the behavior.

[Expected result]:

• The specific telemetry metric “app.opened_as_default_browser” is registered when opening a link from a 3rd party app.

[Actual result]:

• No specific telemetry event or metric is registered in the Glean Debug Ping Viewer.

[Additional Notes]:

• This issue is reproducible on all of the branches from the experiment.

┆Issue is synchronized with this Jira Task

[dnarcese]

The 2 events, default_browser_onboarding.dismiss_pressed; default_browser_onboarding.go_to_settings_pressed are unrelated to first-run onboarding so that is okay that they are not firing.

It seems the only bug here is that app.opened_as_default_browser is not sending. The team can investigate.

[alicevarro]

Thanks for investigating that! I included default_browser_onboarding.dismiss_pressed and default_browser_onboarding.go_to_settings_pressed metrics because the brief is mentioning these as being relevant for this experiment. Could you and your team decide if these metrics are in scope for the experiment, please? Thank you!

[data-sync-user]

➤ Nishant Bhasin commented:

Sorin Paraipan Please timebox it to 1~2days max and sync with Daniela Arcese for any clarifications

This is slightly higher priority for v121 (dot release)

[dnarcese]

@alicevarro is it easy for you to verify whether app.opened_as_default_browser is being sent in the typical use case outside of onboarding? We are trying to isolate if this is broken everywhere or just in this case.

[alicevarro]

Yes, I'll take a look and let you know the outcome tomorrow morning (our time).

[alicevarro]

I have verified that the "app.opened_as_default_browser" metric is successfully sent in Glean without being enrolled in the experiment using Firefox for iOS 121.0 (Build 36383 from TestFlight) and Firefox for iOS 120.2 (Build 36408 from TestFlight). However, I was not able to trigger this metric using Firefox for iOS 121.0 (Build 36255 from TestFlight). While being enrolled in the "Social Proof on iOS "Default Browser" onboarding screen" experiment I was not able to trigger it on any of the mentioned builds.

[data-sync-user]

➤ Sorin Paraipan commented:

Same comment as for: https://mozilla-hub.atlassian.net/browse/FXIOS-7848 ( https://mozilla-hub.atlassian.net/browse/FXIOS-7848|smart-link )

[abodea]

Hello, please note that I was able to generate the app.opened_as_default_browser by opening the app a few times like it's mentioned in the Notes of the event but not in the Description, also note that I was enrolled in the experiment and I did set Firefox as default from the onboarding card. Here you can find the event in glean dashboard.

 "counter": {
      "app.opened_as_default_browser": 3,

Please note that based on the https://dictionary.telemetry.mozilla.org/apps/firefox_ios/metrics/app_opened_as_default_browser

• Notes: Note again that this probe will NOT reflect users who DO have Firefox for iOS set as default but only open the app directly on a given day

• Description: Counts the number of times the app is opened from an external link, implying the client has Firefox set as a default browser. Currently this is our most accurate way of measuring how often Firefox is set as the default browser.

so it might be worth to re-check this event and see 100% how it should be triggered because the description says one thing and how it should be triggered Counts the number of times the app is opened from an external link but the Notes below in the event page says Note again that this probe will NOT reflect users who DO have Firefox for iOS set as default but only open the app directly on a given day (that is, do not open it via an external link). and the part from the Notes being the only way to trigger app.opened_as_default_browser

@dnarcese @afurlan-firefox is there we can do to check this so we know exactly how the 'trigger" method for the event should be, like that we will know 100% if it works correctly or not.

[data-sync-user]

➤ Alina Moldovan commented:

Alice Varro is there anything we can do for this ticket or can we close it?

Thank you!

[data-sync-user]

➤ Alice Varro commented:

Alina Moldovan it can be closed, a sign-off was sent for this experiment already. Thanks!

[data-sync-user]

➤ Alina Moldovan commented:

Closing this thicket based o the previous comment!