Engagement Time is logged incorrectly when disable screen view tracking

[chalermpong]

[READ] Step 1: Are you in the right place?

Yes.

[REQUIRED] Step 2: Describe your environment

• Android Studio version: Android Studio Arctic Fox | 2020.3.1 Patch 3
• Firebase Component: Analytics
• Component version: 20.0.1 (Firebase BoM 29.0.2)

[REQUIRED] Step 3: Describe the problem

This issue is related to my previous issue #3039 Firebase Analytics logs incorrect engagement time. As shown in Log below

2021-12-14 13:42:21.554 16014-16064/my.app.package I/FA: App measurement initialized, version: 46000
2021-12-14 13:42:21.554 16014-16064/my.app.package I/FA: To enable debug logging run: adb shell setprop log.tag.FA VERBOSE
2021-12-14 13:42:21.554 16014-16064/my.app.package I/FA: Faster debug mode event logging enabled. To disable, run:
adb shell setprop debug.firebase.analytics.app .none.
2021-12-14 13:42:21.554 16014-16064/my.app.package D/FA: Debug-level message logging enabled
2021-12-14 13:42:21.580 16014-16064/my.app.package V/FA: Connecting to remote service
2021-12-14 13:42:21.590 16014-16064/my.app.package V/FA: Connection attempt already in progress
2021-12-14 13:42:21.595 16014-16064/my.app.package V/FA: Connection attempt already in progress
2021-12-14 13:42:21.596 16014-16064/my.app.package D/FA: Connected to remote service
2021-12-14 13:42:21.596 16014-16064/my.app.package V/FA: Processing queued up service tasks: 3
2021-12-14 13:42:26.625 16014-16064/my.app.package V/FA: Inactivity, disconnecting from the service
2021-12-14 13:42:50.656 16014-16055/my.app.package V/FA: onActivityCreated
2021-12-14 13:42:50.700 16014-16064/my.app.package V/FA: Activity resumed, time: 246386362
2021-12-14 13:42:50.701 16014-16055/my.app.package V/FA: Logging screen view with name, class: main, MainActivity1
2021-12-14 13:42:50.705 16014-16064/my.app.package I/FA: Tag Manager is not found and thus will not be used
2021-12-14 13:42:50.728 16014-16064/my.app.package V/FA: Connecting to remote service
2021-12-14 13:42:50.745 16014-16064/my.app.package V/FA: Connection attempt already in progress
2021-12-14 13:42:50.799 16014-16064/my.app.package D/FA: Connected to remote service
2021-12-14 13:42:50.802 16014-16064/my.app.package V/FA: Processing queued up service tasks: 2
2021-12-14 13:42:55.810 16014-16064/my.app.package V/FA: Inactivity, disconnecting from the service
2021-12-14 13:42:56.270 16014-16064/my.app.package V/FA: Activity paused, time: 246391932
2021-12-14 13:42:58.272 16014-16064/my.app.package D/FA: Application going to the background
2021-12-14 13:42:58.283 16014-16064/my.app.package V/FA: Recording user engagement, ms: 34745

Explanation for the log above

I kill the app before start of the log above.
13:42:21.554 <-- I sent a data message to the app
13:42:50.700 <-- I started the app
13:42:56.270 <-- I left the app using Android home button

The engagement time should be around 6 seconds, but Firebase Analytics logged 34745 milliseconds. Seems like FA started logging engagement time when app received a data message.

Steps to reproduce:

Prerequisite: The app must disable screen view tracking, and be able to receive data message

1. Start app and kill the app quickly before FA logs engagement time (explanation in note below).
2. Send a data message to the app
3. Open app again
4. Stay in the first screen for a few seconds then leave the app. FA will log engagement time = time since it received the data message instead of time since app open.

Note: When disable screen view tracking, there will be ~2 seconds delay before FA logs engagement time. So it is possible to kill the app quickly before Recording user engagement happens.

2021-12-14 13:42:56.270 16014-16064/my.app.package V/FA: Activity paused, time: 246391932
2021-12-14 13:42:58.272 16014-16064/my.app.package D/FA: Application going to the background
2021-12-14 13:42:58.283 16014-16064/my.app.package V/FA: Recording user engagement, ms: 34745

If the app is not disable screen view tracking, the Recording user engagement, ms: ... will happen before Activity paused, time: ...

[google-oss-bot]

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

[argzdev]

Hi @chalermpong, thanks for reporting. I'll try to check this and see what I can find. In the meantime, if you can provide us with an MCVE, that'll greatly help us with the investigation. Thanks!

[chalermpong]

@argzdev Here is the code that can reproduce this issue. https://github.com/chalermpong/FirebaseEngagementTime

[argzdev]

Thanks for the MCVE, @chalermpong. I was able to repro the issue, I'll notify an engineer and see what we can do here.

[argzdev]

Our engineers have been notified and will look into this. This is now internally tracked on b/210863612. Thanks!

[OksanaAxonista]

Hello, I have the same issue. Since 26 August our engagement times started to rise significantly only on Android It happens with every screen. Here are screenshot from GA for splash.

[mana-break]

I have the same problem. In August 2021 I disabled google_analytics_automatic_screen_reporting_enabled and my in-app engagement time doubled. I would like to know if this is a bug or a consequence of using the google_analytics_automatic_screen_reporting_enabled flag?

[woodii]

We have the exact same issue. We implemented manual screen tracking on both our apps (android and ios) but in the android app the engagement time (average and per session) skyrocketed by 700% from that release on.

[tompesman]

Hi, same problem here. We've compared to our previous version and to iOS.

Can we expect this to be solved or should we enable automatic screen reporting to get our engagement stats back?

Cheers!

[chalermpong]

@tompesman I posted my workaround here. Let's see if it's work on your project.

[tompesman]

That's an interesting one, are you sure that there are no side effects?

[chalermpong]

@tompesman You are right. I'm not sure if there is any side effect.

@argzdev Could you help us confirm if there are no side effects from using this workaround?

Instead of adding this line in AndroidManifest.xml,

<meta-data android:name="google_analytics_automatic_screen_reporting_enabled" android:value="false" />

I used setAnalyticsCollectionEnabled in Activity's onResume to disable automatic screen tracking as shown in code below.

override fun onResume() {
    FirebaseAnalytics.getInstance(this).setAnalyticsCollectionEnabled(false)
    super.onResume()
    FirebaseAnalytics.getInstance(this).setAnalyticsCollectionEnabled(true)
}

[argzdev]

Hi all, thank you for the inputs! Unfortunately, I can't verify if this workaround has no side effects.

I have escalated this to our engineering team, and it looks like our engineers are still investigating this along with other issues and tasks. Given that their hands are full at the moment, it my take some time for them to dig deeper in this issue. That being said, while I’m unable to promise any timeline for this, adding a thumbs up on this issue can help our engineers prioritize this based on the demand and severity.

[leontodd]

Also having the same issue, which makes engagement statistics useless.

@argzdev How do I thumbs up an issue on GitHub? Do you mean using the reaction emojis?

[argzdev]

Yes @leontodd, apologies if that was confusing.

[jaumebd]

Hi, @argzdev there's any update on this issue? I'm experimenting the same misbehaviour after disabling automatic screen reporting

[Julian-Cao]

Are there some new updates？ This issue is still on our apps.

[mypplication]

Hello, Same problem here...

[TaylorPringleNZME]

Hi, Is there any update on this issue? @argzdev

[argzdev]

Hi @TaylorPringleNZME, thanks for following up on us. Looking into the internal bug, it seems that there's a fix on going, though I'm not sure if this has been merged on the latest version. I'll try to get some updates from our engineers. But if possible, please do try using our latest version to see if the fix has been applied. Thanks!

[TaylorPringleNZME]

Hi @TaylorPringleNZME, thanks for following up on us. Looking into the internal bug, it seems that there's a fix on going, though I'm not sure if this has been merged on the latest version. I'll try to get some updates from our engineers. But if possible, please do try using our latest version to see if the fix has been applied. Thanks!

Hi @argzdev, is there any way we can track the fix? Thanks for the update

[argzdev]

Sorry @TaylorPringleNZME, currently the bug details are only available internally, but feel free to ping on any threads whenever you'd like to get an update, and we'll take a look into it.

Our engineers have informed us that the fix for this issue is being rolled out right now. There is no need to update the SDK version for an app unless the app is being distributed on non-Play devices.

Full rollout is expected by the end of this week.

[TaylorPringleNZME]

Hi @argzdev, thank you for the update.

Are you able to confirm if the fix was fully rolled out last week please? Also when would we expect to see a change in results?

[debaerebart]

Hi all,

I am writing down an analytics solution for a client of ours and I'm stuck on below question (and can't seem to find a clear answer anywhere):

Does your engagement time still get logged if you disable the automatic screen view tracking?

The plan is to implement the screen_view event ourselves, however in the example code the user_engagement_msec is not added to the custom event (https://firebase.google.com/docs/analytics/screenviews).

So, I am wondering is that intentional or is it just to keep the example simple?

I bumped into this post, and I know my question is not entirely on topic, but it thought that somebody in this audience might know the answer.

Appreciate it if somebody can give a clear answer!

[argzdev]

Yes @TaylorPringleNZME, the fix should be fully rolled out by now. You should start seeing a change in results immediately.

@debaerebart it would be best if you reach out to our Firebase support team. They'll be able to clarify any questions you may have.

That said, since this issue have been resolved. I'll close this now. If anyone experiences this issue again, just let me know and we'll reopen this thread. Thanks!

[woodii]

@argzdev are you sure this will take effect immediately? I don't see any changes on our side.

[mthormann-dhi]

Same here. No change at all.

[argzdev]

Hi @woodii, it should have improvements compared to the past few days but clearly this doesn't look like it. Let me get some feedback from our engineers. Thanks for checking!

If others have a different finding/behavior, let me know. Thanks!

[TaylorPringleNZME]

Hi @argzdev, we have also not seen any changes in the past few days.

[lbarrettanderson]

@argzdev are you sure this will take effect immediately? I don't see any changes on our side.

@woodii The fix is at 50% rollout, so by now you should see a significant improvement. Since you aren't, can you share with me your package name so I can research further?

[jmartinMone]

@argzdev @lbarrettanderson Hello, no changes here.

one.manager.app auto tracking disabled in Android/iOS

[lbarrettanderson]

Thank you that's really helpful.

Based on a quick look, I do see a slight improvement between the two experiment arms, but clearly not as much as expected. It seems the fix only addresses one cause of the issue. I'll investigate further.

[woodii]

Hello @argzdev @lbarrettanderson thanks for taking care of this issue. Still no change in logged times. at.wienerlinien.wienmobillab is our biggest app.

If you need any further information or implementation details, let me know.

[lbarrettanderson]

Thank you. I'm investigating. If you are able to reproduce this issue without killing the app process, please let me know what steps you're taking. The fix that's currently at 50% rollout seems to fix the repro steps described in this issue's OP.

[woodii]

Today I had some time to look into this again. Unfortunately I coudln't find any hint, but maybe my research helps.

If i check the analytics logs via

adb shell setprop log.tag.FA VERBOSE
adb shell setprop log.tag.FA-SVC VERBOSE

I couldn't find any faulty "recording user engagement" entries. All measurements look fine.

But if I check my engagement reports all (not just some screens) have way higher engagement times than they should have.

[ninniuz]

We are having the same issue on different apps where automatic screen view tracking has been disabled.

I don't see any odd values for user engagement in logcat but reports are completely out of scale now.

Do you have any updates?

[Eluch]

Same issue here. Pretty huge difference between iOS and Android AppId: hu.mol.move And because the app basically "scan the loyalty card and forget the app", the 1 minute average engagement time seems reasonable.

[rebeccaoskarsson]

We also experience the exact same issue. We have done the update including a fix for this bug, but no change. We are considering switching to Snowplow due to this. Any updates on this issue before we start working with a different analytics tool?

[jmartinMone]

"Ok Google, can you explain this issue ?" ;-)

@lbarrettanderson @argzdev

[alecoopz]

Hello!

We are experiencing the same issue after disabling screen view tracking on 2 apps. It seems to happen when users do not close the app and it runs on background. Once in a while the event user_engagement fires, renewing the session. This issue impacts all GA4 Engagement and some Session metrics (only in Android platform).

Enabling automatic screen view tracking is not a solution for us.

[ilhamiprospect]

Hi, I have the same issue. Kindly please consider this a serious problem @argzdev

Thank you

[rajeshjadidminc]

Hi, I have the same issue with Firebase SDK BOM version: 32.5.0 with Jetpack Compose.

Is there any update on this issue?

Thanks!

[argzdev]

Hi folks, sorry for the radio silence here. I've been trying to get in touch with our engineering team and request for updates. So far our engineers are aware of this issue and have been keeping this issue on their priority list. However, they're also occupied with other tasks at hand, so things are taking longer to figure out. Rest assured, we'll share updates once we have progress in our findings.

[michalnovacek96]

Hi @argzdev Please any updates on this?

Almost same issue here. But we did NOT disable automatic screen_view tracking, but beyond automatic screen_view tracking we are tracking manually screen views.

[woodii]

@argzdev any updates on this? I am really curious that such a major issue which makes analysis of your project much harder or even pointless has such a low priority for over 2 years. Is there anything we can do to help you fix this?

[alecoopz]

@argzdev Hello, do you have any update? We are GA 360 users and from what we've analyzed this event is consuming a big percentage of our monthly event contract. By the way, we're considering to change to another tool in the next year because of this experience. There is any way to prioritize this issue?

[lbarrettanderson]

@argzdev Hello, do you have any update? We are GA 360 users and from what we've analyzed this event is consuming a big percentage of our monthly event contract. By the way, we're considering to change to another tool in the next year because of this experience. There is any way to prioritize this issue?

Can you share with me your App ID(s)? We're looking into this again.

[woodii]

@lbarrettanderson 1:165995409996:android:334b06cecc30a1ba Thanks! Please let me know if we can help in any other way

[alecoopz]

@argzdev Hello, do you have any update? We are GA 360 users and from what we've analyzed this event is consuming a big percentage of our monthly event contract. By the way, we're considering to change to another tool in the next year because of this experience. There is any way to prioritize this issue?

Can you share with me your App ID(s)? We're looking into this again.

Of course, here it is: 1:590874711807:android:52ff104a705b789d

[lbarrettanderson]

The fix is rolled out. I've checked and I'm seeing significant improvement as of yesterday and today. I expect the improvement will continue and your numbers will look more or less reasonable by early next week.