Investigate Options for Experiment Where App Icon Background Colors Are Different

[data-sync-user]

Hypothesis: A white app logo background makes the FXIOS app easier for users to find on their home screens rather than a dark purple app logo background.

Problem:

We would like to understand the best way to create and deliver an experiment where the TEST group gets a white logo background and the CONTROL group gets the dark purple (as-is).

Metrics we would like to track a number of sessions opened per month, and retention for both groups.

Ideally, the TEST and CONTROL groups are a subset of the total population and both can be randomized from the total population so we can achieve statistical significance.

┆Issue is synchronized with this Jira Spike

[data-sync-user]

➤ Roux Buciu commented:

Jeremy Evans re: iOS leads covresnaiton

[data-sync-user]

➤ Matt Reagan commented:

I did a quick dev spike on this, findings are summarized below. TL;DR: this should be a relatively low LOE to implement. However, one issue I wanted to call out is that iOS shows an alert to the user whenever we programmatically change the app icon.

Potential Issues

• When changing the app icon, an alert is presented to the user (attached). There may be potential hacks to work around this (this link ( https://medium.com/@diananareiko8/two-methods-for-avoiding-alert-display-when-changing-icons-in-ios-d74e35d1126c ) discusses 2 options, the latter of which is safer but could still be prone to breakage in future OS updates); I haven't yet tested these but happy to explore them further.
• The alternative icon cannot be set immediately during app launch. This appears to be an iOS limitation (I found reports from other devs experiencing similar problems). If we attempt to change the icon too early it will result in an operation cancelled error. Changing the code to update the icon after a brief delay fixed the problem for me and worked reliably from then on.

Implementation Summary

1. Icons must be declared in the Info.plist of the client, for the CFBundleIcons key (example entry below)
2. Icon image file must be added to the project "loose" (not in the asset catalogue)
3. We check supportsAlternateIcons on UIApplication, and if it returns true (which it should if we have configured it correctly) then we call setAlternateIconName to change the icon
4. We can revert the icon to the default app icon by passing nil as the argument for the above function

Feature Flag

• This aspect should be straightforward; we can place this code programmatically behind a Nimbus feature flag

Metrics / Telemetry

• I believe that we should be able to handle all of the necessary data collection on the backend without any significant changes to metrics client-side, by cross-referencing the IDs of users in the experiment group against the retention data we’re interested in (app session length, frequency, etc.).

Other Notes

• The APIs for the icon change do not detect “redundant” updates. IOW: if we set the same alternative icon repeatedly, even if it’s already set, iOS will still show the alert. So we would probably want to leverage a user default or other persistent storage mechanism to make sure we avoid that.
• Dev/Beta/App store icons: AFAICT this experiment should not create any conflicts or issues for the custom icons we use to differentiate the Beta/App store/Developer builds. I did a quick test by just building both Fennec and Firefox and could see the standard/expected icon, and the above code updated the icon as expected, and then I was able to again revert it back to the standard version by setting it back to nil.
• The UIPrerenderedIcon value for the Info.plist allows us to disable the “shine” that is applied by the OS on the home screen, if we have such an effect already in the icon’s visual design. Since we don’t, the value should be false. (Reference ( https://developer.apple.com/documentation/bundleresources/information_property_list/uiprerenderedicon ))

Example Info.plist entry (where the alternative icon is named AppIconAlt in this case):

CFBundleIcons CFBundlePrimaryIcon CFBundleIconFiles AppIcon UIPrerenderedIcon CFBundleAlternateIcons AppIconAlt CFBundleIconFiles AppIconAlt UIPrerenderedIcon cc gangadi Orla Mitchell Norberto Andres Furlan Jeremy Evans !Screenshot 2024-05-24 at 2.04.53 PM.png|width=338,height=183,alt="Screenshot 2024-05-24 at 2.04.53 PM.png"!