Device changes to previous "cio_sdk_version"

[TwistedMinda]

SDK version: 3.3.1

Environment: Production

Are logs available?

cio_sdk_version: {
  from: "3.3.1"
  to: "2.9.2"
}

Describe the bug Weirdly enough, our users on the new version of CustomerIO see their device having weird logs, they jump from cio_sdk_version 2.9.2 to 3.3.1 and vice-versa. We believe this is the root cause of our users sometimes receiving more than 1 push notification for 1 broadcast.

To Reproduce To reproduce you first need to identify a user with 2.9.2 version of CustomerIO, then without uninstalling app update the CustomerIO SDK to 3.3.1

Expected behavior We would expect that the last version takes precedence and also takes care of removing old device tokens related to an old version of the SDK

Additional context More than anything, we'd like to understand what's going on here, so we can decide how to best handle that case and find out whether it's indeed the root cause of some of our users receiving push notifications multiple times (duplicated).

• Duplication of the notifications only happens with Broadcasts!
• Some of our users have 2 device tokens, due to the version bug mentioned (each one has a different cio_sdk_version but point to the same device)
• Duplication actually also happen for users with only 1 device token
• Sending directly to specific device tokens or a campaign event never creates a duplication

[mrehan27]

Hi @TwistedMinda, Thank you for sharing the details. Sorry to hear about the issue you're facing. Let me respond to your questions one by one.

The version jumps you're seeing from 3.3.1 to 2.9.2 occur due to SDK initialization in the iOS native code for handling push metrics. Here, 3.3.1 is the React Native package version and 2.9.2 is the iOS SDK version. This typically happens when the app was killed before launch. A few cases have been reported where you might see duplicate push metrics in the activity log. However, this doesn't cause notifications to be displayed multiple times. After the first request post-launch, the SDK version will be set to the React Native SDK and won't update further in that session. We're aware of this issue and it's on our roadmap. However, a fix may not be approaching very soon due to the impact its resolution could have on the rest of the SDK and other tasks currently taking priority.

Regarding notification duplication, we haven't had any reports of it being linked to this issue. Are you inferring this from multiple delivered metrics in the Activity Logs? Or were there instances of multiple notifications on an actual device? If you know steps to reproduce this, or can share logs to review, that will help us identify the issue better.

As for users with two device tokens, do these users have only one device but appear twice on dashboard? If you're using FCM for iOS, FCM initially provides an old token, which is updated with a new one provided by FCM after some time. Due to this, the same device might appear twice because FCM issued two different tokens for it. Once you send a notification, the older token will be invalidated and removed, ensuring the user receives only one notification with the newer token.

I hope this helps. Let me know if you have more questions. Happy to assist. Have a great day!

[TwistedMinda]

Thanks @mrehan27 for this quick response!

Okay so from what I understand the logs we're seeing are not specific to us, and should not cause notifications to appear multiple times. It shouldn't be the root cause of our issue, that's good news ;)

Are you inferring this from multiple delivered metrics in the Activity Logs?

Yes! We also have users reaching out to us with screenshots proving that they have notification duplication (more than two) and we can see that they have multiple "Delivered" in activity logs, exactly matching the number of notifications they see on their device. So we concluded that this is the root cause.

We only see this behaviour when using a broadcast by the way, could this be an internal customerIO behaviour that we are not aware of?

As for users with two device tokens, do these users have only one device but appear twice on dashboard? If you're using FCM for iOS, FCM initially provides an old token, which is updated with a new one provided by FCM after some time. Due to this, the same device might appear twice because FCM issued two different tokens for it. Once you send a notification, the older token will be invalidated and removed, ensuring the user receives only one notification with the newer token.

Unfortunately it seems that only iOS users are having the problem, but I can't say that for sure, only that all customers reaching out to us are iOS users, after checking again.

Finally, the devices don't seem to delete themselves and are indeed still being used when a broadcast is sent, which is to me the reason for the duplication. But what loses me is that some users don't have this kind of setup with multiple devices and still receive the notification multiple times (again, only from broadcasts).

Here is an example of devices accumulating and not deleting themselves as the customerIO versions go up

Do you confirm that we shouldn't be deleting tokens ourselves?

To summarize the key points:

• logs of versions switching should not be a problem for our issue
• issue only happens for broadcasts
• multiple devices are accumulated for some users, so they receive the notification duplicated for any broadcast
• some users have only 1 device token but still receive the notification multiple times
• our issue is probably only related to iOS
• we did not manage to make the Rich Push setup work both on iOS and Android which could be a reason of "bad setup" leading to this kind of bugs

Thanks again for your time

[mrehan27]

Thanks a lot for providing all the details, they're really helpful.

We also have users reaching out to us with screenshots proving that they have notification duplication

This is unusual and something we haven't encountered before. Is it possible for you to identify those users and share their profiles with us? We'd like to examine their activity and notification logs to debug this issue further.

The multiple notifications issue is indeed worrisome for us as well. Sharing a few of these user profiles at win@customer.io (including a link to this ticket) would greatly help us in identifying the root cause.

Do you confirm that we shouldn't be deleting tokens ourselves?

If possible, I would ask not to delete them so we can investigate first and determine why they weren't automatically removed.

Regarding your key points:

logs of versions switching should not be a problem for our issue

That's correct.

we did not manage to make the Rich Push setup work both on iOS and Android, which could be a reason for "bad setup" leading to these kinds of bugs

Can you please share more details about this? Do you have a custom setup for managing push notifications? It would be helpful if you could share some relevant files like AppDelegate and NotificationServiceExtension via email.

Also, can you confirm whether you're using APN or FCM for iOS?

Please feel free to reach out to us here for general queries. However, please ensure that any non-public information is communicated through win@customer.io.

[TwistedMinda]

If possible, I would ask not to delete them so we can investigate first and determine why they weren't automatically removed.

Yep, we wanted to proceed that way too

Sharing you via email the AppDelegate and NotificationServiceExtension as well as the user accounts that consistently receive multiple notifications when we send a broadcast.

Also, can you confirm whether you're using APN or FCM for iOS?

We're using APNs for iOS and FCM for android.

Thanks for your strong support!

[mrehan27]

Great, thank you @TwistedMinda. Let's continue the discussion via email. We'll reach out to you soon after receiving the updates. Have a great day.

[levibostian]

Closing to continue helping via email.