Exclude internal users from Firebase statistics

[wjt]

We discussed a few ways to do this today, in no particular order:

• Hardcode a list of Google account IDs or device IDs
• Use the Google Play track ("internal testing" group vs public streams) if possible
• Add a manually-controlled flag that can be set before installing & launching the app
• Disable Firebase integration in internal builds (with a way to override)
• ...

[dbnicholson]

I think I have a good story going from the debug build side and hacked it together yesterday.

First, you can use a manifest tags for analytics and crashlytics to disable collection by default. You can then use separate manifest placeholders in the debug and release buildTypes so that they're enabled by default for release builds and disabled by default for debug builds. This needs a bit of p4a support to marshal through the data you want, but it's achievable.

In the event that you want to enable analytics for debugging (when hacking on analytics, for example), you can actively enable them again at runtime using setAnalyticsCollectionEnabled and setCrashlyticsCollectionEnabled. The trick is knowing when to do that.

What I've come up with is using the BuildConfig class generated by gradle that includes a boolean DEBUG constant to see when this is a debug build. In that case, we can use a debug boolean system property accessed with the android.os.SystemProperties API as the switch. These are the properties controlled with adb shell {get,set}prop. Then developers can run adb shell setprop debug.org.endlessos.Key.analytics {true,false} to turn on analytics as needed.

Putting it all together, the code looks like this:

    if BuildConfig.DEBUG:
        analytics_enabled = SystemProperties.getBoolean(
            "debug.org.endlessos.Key.analytics",
            False,
        )
        logging.debug(
            f"Debug build, debug.org.endlessos.Key.analytics = {analytics_enabled}"
        )
    else:
        analytics_enabled = True

    # Analytics and Crashlytics collection enablement persists across
    # runs, so actively enable or disable based on the current settings.
    logging.info(
        "%s Firebase Analytics and Crashlytics",
        "Enabling" if analytics_enabled else "Disabling"
    )
    context = get_activity()
    analytics = FirebaseAnalytics.getInstance(context)
    crashlytics = FirebaseCrashlytics.getInstance()
    analytics.setAnalyticsCollectionEnabled(analytics_enabled)
    crashlytics.setCrashlyticsCollectionEnabled(analytics_enabled)

[dbnicholson]

https://github.com/endlessm/python-for-android/pull/18 adds support for using manifest placeholders specific to build types. After that we can add the following to .p4a:

--debug-manifest-placeholders '[analytics_enabled: "false"]'
--release-manifest-placeholders '[analytics_enabled: "true"]'
--meta-data firebase_analytics_collection_enabled=${analytics_enabled}
--meta-data firebase_crashlytics_collection_enabled=${analytics_enabled}

[dbnicholson]

We discussed a few ways to do this today, in no particular order:

• Hardcode a list of Google account IDs or device IDs

Obviously, this would be a last resort.

• Use the Google Play track ("internal testing" group vs public streams) if possible

From everything I can tell, this isn't possible. While the Play Store app clearly knows what track to look at, I believe all it does is use it for version selection. Once the app is installed, I don't think that data is stored anywhere. It doesn't show up in App Info and people assert it's not possible to get this data. Crashlytics has a feature where it can filter crashes by Google Play track, but in my testing all it does is provide a convenient filter to choose the current version for a particular track.

• Add a manually-controlled flag that can be set before installing & launching the app

After digging around for a while, it was able to find how to read system properties (what you manage with adb shell {get,set}prop) at runtime. On my emulator in Android API 31 it appears that the debug. namespace can be freely written to, so #132 uses a boolean debug.org.endlessos.key.analytics property to actively enable or disable.

• Disable Firebase integration in internal builds (with a way to override)

132 disables analytics by default on debug builds. Combined with the above system property, you can explicitly enable it to test the analytics itself.

The other idea I had was to always have Endless testers install from the Play Store using a referrer like described in https://phabricator.endlessm.com/T34783. For example, https://play.google.com/store/apps/details?id=org.endlessos.Key&referrer=utm_source%3deosinternal%26utm_campaign%3dtest or something like that. Then we can filter out events in Google Analytics by the test campaign. We can also use the Install Referrer API to disable at runtime, but this would still let through the events sent from the Java side early in the application lifecycle. Those are the events that are built into the acquisition graphs. Still, disabling at runtime would mean any custom events we added wouldn't need to be filtered.

[dbnicholson]

136 documents a URL for Endless testers to use that I'll publicize if it looks OK. It also uses the Install Referrer API to gather the install referrer in the app. It's not actually used for anything at the moment since I want to get that code into a released version so I can see what the data looks like when actually installing from the Play Store. At the moment I can only sideload the app, so I don't really know for sure what the data looks like.

[dbnicholson]

136 got merged, so I'm going to push out a new internal testing release with that and then instruct everyone to please install from https://play.google.com/store/apps/details?id=org.endlessos.Key&referrer=utm_source%3Deos%26utm_campaign%3Dtest.

[dbnicholson]

I uploaded that version as Ladybird 6.1-328 to internal testing and then tried out that Google Play campaign link. After starting I see this in the logs:

05-30 20:57:56.109  8601  8601 I org.endlessos.Key: kolibri_android.referrer: Install Referrer service connected
05-30 20:57:56.117  8601  8719 D org.endlessos.Key: kolibri_android.android_utils: Setting referrer_url='utm_source=eos&utm_campaign=test'
05-30 20:57:56.124  8601  8719 I org.endlessos.Key: kolibri_android.android_utils: Installed from referrer URL 'utm_source=eos&utm_campaign=test'
05-30 20:57:56.125  8601  8719 D org.endlessos.Key: kolibri_android.android_utils: Install referrer parameters: {'utm_source': 'eos', 'utm_campaign': 'test'}

If I install from the Play Store by searching for "endless key", the referrer "URL" is utm_source=google-play&utm_medium=organic. This also appears to be the referrer URL used when sideloading the APK.

[dbnicholson]

This got released to internal testing in Ladybird 6.2-331.