Advertising Id not being sent for Android Application Installed events using analytics-react-native-plugin-advertising-id

maurispalletti commented 2 months ago

Updated issue:

There is an issue in the analytics-react-native-plugin-advertising-id (https://github.com/segmentio/analytics-react-native/tree/master/packages/plugins/plugin-advertising-id) where its functioning defers from the Android native implementation. When the the event Application Installed is triggered, it is missing the advertisingId value.

Why is this happening?

It seems like at this point, where plugins are being added in the forEeach, and since at least AdvertisingIdPlugin is async, the manual flush from this.store.pendingEvents including the event Application Installed is being executed before finishing adding all plugins.

https://github.com/segmentio/analytics-react-native/blob/c1a0957c3678e9ba3678a07ca7af9eaa37b7dc66/packages/core/src/analytics.ts#L478-L508

So, the current process follows this sequence:

this.checkInstalledVersion() is being called here, where Application Installed event is created internally. However, since this.isReady.value is not yet set to true, the event is added to a store of pending events here.
Then await this.onReady() is being called, during which the plugins are being added. Inside a forEach, this.addPlugin(plugin) is executed for each plugin. In the case of the plugin AdvertisingIdPlugin it should be done async since the plugin itself is async. And not synchronously as it's done in the forEeach here.
Finally, the list of pending events is being processed and a manual flush is performed to trigger them here.

So there may be a case where AdvertisingIdPlugin has not been added yet, including its context with an advertisingId and pending events like Application Installed were flushed, resulting in these first events being sent without its corresponding advertisingId.

Proposed fix

Replace: https://github.com/segmentio/analytics-react-native/blob/c1a0957c3678e9ba3678a07ca7af9eaa37b7dc66/packages/core/src/analytics.ts#L482-L493 by:

  try {
    // start by adding the plugins
    for (const plugin of this.pluginsToAdd) {
      await this.addPlugin(plugin);
    }
    // now that they're all added, clear the cache
    // this prevents this block running for every update
    this.pluginsToAdd = [];
  } finally {
    this.isAddingPlugins = false;
  }

Details

analytics-react-native version: 2.17.0
Integrations versions (if used): analytics-react-native-plugin-advertising-id: 1.3.1
React Native version: 0.71.18
iOS or Android or both? Android

Steps to reproduce

Check Application Installed events on Segment.

Expected behavior

All Application Installed events from Android should have advertisingId.

Actual behavior

No Application Installed events from Android actually have advertisingId.

cgadam commented 2 months ago

Thanks for the analysis @maurispalletti ! Yes, it seems that configure in the plugin is of async nature given that clearly it needs to communicate with the native layer. That part can be seen here: https://github.com/segmentio/analytics-react-native/blob/c1a0957c3678e9ba3678a07ca7af9eaa37b7dc66/packages/plugins/plugin-advertising-id/src/AdvertisingIdPlugin.ts#L37

simonwh commented 1 month ago

Any updates on this? @maurispalletti did you manage to find a solution?

cgadam commented 1 month ago

@simonwh we have raised this issue to Segment Support team so they can address this on their end but haven't yet hear anything back from them. If this continues, we will likely try to implement a patch and see if our approach works. Will keep you posted if we do that.

alanjcharles commented 1 month ago

Hi @simonwh @maurispalletti @cgadam- thank you for reaching out and my sincere apologies for the delay. We are still working through some internal turnover/ramping but things should start to be more consistent moving forward as we now have at least one engineer dedicated to this library full time.

With that in mind, we have investigated/tested this issue and definitely understand where you're coming from. However, the issue is not something we plan to fix in the core SDK. Instead, it is important to consider the timing of what is happening. From our investigation, these events do not include the advertisingId value because they are created before consent has been granted by a user. The advertising-id-plugin is of type enrichment which means events are passed through it after they have been created. What we need to do is hold those events in a queue , or before the event timeline starts processing, to wait for a user to grant consent. Once consent has been granted, we can release existing events from the queue and process them accordingly.

As an example, here is a very basic ConsentManagement plugin of type before:

import {
  Plugin,
  PluginType,
  SegmentEvent,
} from '@segment/analytics-react-native';

import type {SegmentClient} from '@segment/analytics-react-native';

import {Alert} from 'react-native';

export class ConsentManager extends Plugin {
  **type = PluginType.before;**
  key = 'Consent Manager';

  consentStatus?: boolean;
  queuedEvents: SegmentEvent[] = [];

  configure(analytics: SegmentClient) {
    this.analytics = analytics;

    this.showAlert();
  }

  execute(event: SegmentEvent): SegmentEvent | undefined {
    if (this.consentStatus === true) {
      return event;
    }
    if (this.consentStatus === undefined) {
      this.queuedEvents.push(event);
      return;
    }
    return;
  }

  showAlert = () => {
    Alert.alert(
      'Consent to Tracking',
      'Do you consent to all of the things?',
      [
        {
          text: 'Yes',
          onPress: () => this.handleConsent(true),
          style: 'cancel',
        },
        {
          text: 'No',
          onPress: () => this.handleConsent(false),
          style: 'cancel',
        },
      ],
      {
        cancelable: true,
        onDismiss: () => (this.consentStatus = undefined),
      },
    );
  };

  handleConsent(status: boolean) {
    if (status === true) {
      this.consentStatus = true;
      this.sendQueued();
      void this.analytics?.track('Consent Authorized');
    }
    if (status === false) {
      this.queuedEvents = [];
    }
  }

  sendQueued() {
    this.queuedEvents.forEach(event => {
      void this.analytics?.process(event);
    });
    this.queuedEvents = [];
  }
}

**note this is a simple example that you are free to use if it works for you but it is technically not something we support. The SDK was designed with a flywheel approach to make it possible for you to customize and this is a great example of when you might want to do that.

if we add this plugin along with the advertising-id-plugin we are able to see the advertising-Id value added every single time Application Installed is invoked.

I have added some additional documentation here in regard to the architecture below for your reference. I hope this helps everyone, I will leave the issue open for now so please do let me know if you have any additional questions/concerns and we can go from there. Thanks again!

https://github.com/segmentio/analytics-react-native?tab=readme-ov-file#plugins--timeline-architecture https://segment.com/blog/analytics-react-native-2-blog/

maurispalletti commented 1 month ago

@Christian Adam @.> and @Ignacio Mendez Nolé @.>

On Mon, Oct 21, 2024 at 9:01 AM Alan Charles @.***> wrote:

Hi @simonwh https://github.com/simonwh @maurispalletti https://github.com/maurispalletti @cgadam- thank you for reaching out and my sincere apologies for the delay. We are still working through some internal turnover/ramping but things should start to be more consistent moving forward as we now have at least one engineer dedicated to this library full time.

With that in mind, we have investigated/tested this issue and definitely understand where you're coming from. However, the issue is not something we plan to fix in the core SDK. Instead, it is important to consider the timing of what is happening. From our investigation, these events do not include the advertisingId value because they are created before consent has been granted by a user. The advertising-id-plugin is of type enrichment which means events are passed through it after they have been created. What we need to do is hold those events in a queue , or before the event timeline starts processing, to wait for a user to grant consent. Once consent has been granted, we can release existing events from the queue and process them accordingly.

As an example, here is a very basic ConsentManagement plugin of type before:

import { Plugin, PluginType, SegmentEvent, } from @.***/analytics-react-native';

import type {SegmentClient} from @.***/analytics-react-native';

import {Alert} from 'react-native';

export class ConsentManager extends Plugin { type = PluginType.before; key = 'Consent Manager';

consentStatus?: boolean; queuedEvents: SegmentEvent[] = [];

configure(analytics: SegmentClient) { this.analytics = analytics;
this.showAlert();
}

execute(event: SegmentEvent): SegmentEvent | undefined { if (this.consentStatus === true) { return event; } if (this.consentStatus === undefined) { this.queuedEvents.push(event); return; } return; }

showAlert = () => { Alert.alert( 'Consent to Tracking', 'Do you consent to all of the things?', [ { text: 'Yes', onPress: () => this.handleConsent(true), style: 'cancel', }, { text: 'No', onPress: () => this.handleConsent(false), style: 'cancel', }, ], { cancelable: true, onDismiss: () => (this.consentStatus = undefined), }, ); };

handleConsent(status: boolean) { if (status === true) { this.consentStatus = true; this.sendQueued(); void this.analytics?.track('Consent Authorized'); } if (status === false) { this.queuedEvents = []; } }

sendQueued() { this.queuedEvents.forEach(event => { void this.analytics?.process(event); }); this.queuedEvents = []; } }

**note this is a simple example that you are free to use if it works for you but it is technically not something we support. The SDK was designed with a flywheel approach to make it possible for you to customize and this is a great example of when you might want to do that.

if we add this plugin along with the advertising-id-plugin we are able to see the advertising-Id value added every single time Application Installed is invoked. image.png (view on web) https://github.com/user-attachments/assets/dc4a0d3f-b8cd-4e78-b71c-6dc5252c50b0

I have added some additional documentation here in regard to the architecture below for your reference. I hope this helps everyone, I will leave the issue open for now so please do let me know if you have any additional questions/concerns and we can go from there. Thanks again!

https://github.com/segmentio/analytics-react-native?tab=readme-ov-file#plugins--timeline-architecture https://segment.com/blog/analytics-react-native-2-blog/

— Reply to this email directly, view it on GitHub https://github.com/segmentio/analytics-react-native/issues/995#issuecomment-2426473974, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACBNJX427DQ3DOOBQYQBJETZ4TUKFAVCNFSM6AAAAABNS5BB22VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRWGQ3TGOJXGQ . You are receiving this because you were mentioned.Message ID: @.***>

-- Mauricio Spalletti Senior Software Engineer | MarTech

--

Please consider the impact on the environment before printing this email.

The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you receive this in error, please contact the sender and delete the material from any computer.

Diese E-Mail einschließlich evtl. angehängter Dateien enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind und Sie diese E-Mail irrtümlich erhalten haben, dürfen Sie weder den Inhalt dieser E-Mails nutzen noch dürfen Sie die evtl. angehängten Dateien öffnen und auch nichts kopieren oder weitergeben/verbreiten. Bitte verständigen Sie den Absender und löschen Sie diese E-Mail und evtl. angehängte Dateien umgehend.

cgadam commented 1 month ago

@alanjcharles thanks for your detailed response. We are aware that we need ATT consent for iOS, and that's not a problem, but this is still happening in Android were AFAIK permissions are by default granted. Actually the issue that was reported to us is related to Android platform.

alanjcharles commented 1 month ago

Hi @cgadam understood, thanks for the additional context. I am investigating a better fix for this but in the meantime the overall point about the architecture stands.

If you need a solution as we continue our investigation, you can use a .before plugin to hold events generated natively (basically copy the consent plugin i shared but remove the pop-up alert logic and make some other minor changes). You can use the analytics.adTrackingEnabled watchable to check for the value and once you have it, you can process any events that are in the queue.

I will follow up later today or tomorrow with more information related to any changes we are making on our end to resolve this. Thanks again!

ignaciomendeznole commented 1 month ago

Hey @alanjcharles !

Thank you for your previous response and the insights you provided. I’ve been working through two potential solutions for handling the advertising ID on Android, and I wanted to clarify a few things based on our use case:

Scenarios We’re Facing:

Custom Plugin with Queued Events:

If the app is installed with adTrackingEnabled set to false, events like "Application Installed" are queued. However, if the user later enables adTrackingEnabled while the app is backgrounded, the onChange method is not triggered, leading to the loss of the queued events. When the app is reopened, only new events like "Application Opened" are sent with the advertising ID, but the "Application Installed" event is lost. Async/Await Solution:

In this scenario, events are sent immediately, even if adTrackingEnabled is false. While this avoids queuing issues, it means that the "Application Installed" event and others are sent without the advertising ID. Only subsequent events (after adTrackingEnabled becomes true) include the advertising ID. Given these challenges, the async/await approach seems more promising as it ensures events are sent without delay, but we'd prefer to avoid sending important events like "Application Installed" without the advertising ID.

Could you provide a more detailed example of how to implement adTrackingEnabled.onChange in the plugin to ensure that we capture the advertising ID properly for all events, including when the app is backgrounded or reopened? Specifically, we’d like to understand how to persist and process events reliably when adTrackingEnabled changes while the app is in the background or after it’s reopened.

This is how the Plugin looks like right now:

import {
  Plugin,
  PluginType,
  SegmentEvent,
} from '@segment/analytics-react-native';
import type { SegmentClient } from '@segment/analytics-react-native';

export class AdvertisingIdManager extends Plugin {
  type = PluginType.before; // Plugin type 'before', meaning it runs before events are processed.
  key = 'AdvertisingIdManager'; // Identifier for the plugin.

  trackingEnabled?: boolean; // Tracks the state of ad tracking (enabled or not).
  queuedEvents: SegmentEvent[] = []; // Stores events that need to be queued before ad tracking is enabled.

  configure(analytics: SegmentClient) {
    this.analytics = analytics;

    this.analytics.track('Ad Tracking Enabled Configure', {
      enabled: this.trackingEnabled,
    });

    // Watch the `adTrackingEnabled` value and update the plugin state based on its value.
    analytics.adTrackingEnabled.onChange((enabled) => {
      this.trackingEnabled = enabled;
      void this.analytics?.track('Ad Tracking Enabled Change', { enabled });
      if (this.trackingEnabled === true) {
        // Once tracking is enabled, process the queued events.
        this.sendQueued();
      }
    });
  }

  // This method intercepts the event and checks whether tracking is enabled.
  // If tracking is enabled, it processes the event normally.
  // If not, it queues the event for later.
  execute(event: SegmentEvent): SegmentEvent | undefined {
    if (this.trackingEnabled === true) {
      // If tracking is enabled, allow the event to proceed.
      return event;
    } else {
      // If tracking is not enabled, queue the event for later processing.
      this.queuedEvents.push(event);
      return; // Stop the event from continuing through the pipeline.
    }
  }

  // Send all the queued events when tracking is enabled.
  sendQueued() {
    this.queuedEvents.forEach((event) => {
      void this.analytics?.process(event); // Process each queued event.
    });
    this.queuedEvents = []; // Clear the queue once all events are processed.
  }
}

Thanks! 😄

ignaciomendeznole commented 1 month ago

@alanjcharles I've tried also adding an await to the for loop that's in charge of injecting the plugins, but does not seem to work either (still getting no advertising ID in spite of having adTrackingEnabled = true). Please let me know if you have more insights, but it's becoming super complex at this point and I don't have a full understanding of the entire plugin injection flow.

alanjcharles commented 1 month ago

Hi folks- we added a fix for this in 1.3.2 and just deployed a release. I'm going close this issue out for now but please feel free to follow up if you are still running into problems and we can re-open and take another look. Thanks for your patience here!

ignaciomendeznole commented 1 month ago

Hey @alanjcharles ! Thanks for fixing it! Quick question, I just triggered some events with Advertising ID privacy setting turned off, and no events came in at all 🤔 (I was expecting events coming in but without advertising ID). Then, I went ahead and enabled advertising ID, but the app did not trigger any change on the ad tracking enabled state, and events were lost until I closed the app and re-ran it. Is this an expected behavior?

ignaciomendeznole commented 1 month ago

Hey @alanjcharles ! Thanks for fixing it! Quick question, I just triggered some events with Advertising ID privacy setting turned off, and no events came in at all 🤔 (I was expecting events coming in but without advertising ID). Then, I went ahead and enabled advertising ID, but the app did not trigger any change on the ad tracking enabled state, and events were lost until I closed the app and re-ran it. Is this an expected behavior?

Also, LimitAdTrackingEnabled (Google Play Services) is enabled is not triggered anymore when ad tracking is disabled 💭

ignaciomendeznole commented 1 month ago

If the plugin is of type enrichment, is it supposed to hold off events in the queue until they can be processed? I thought it was only in charge of adding additional information to the event, and if that information was not available for whatever reason, then the event was processed anyways but without the enrichment

segmentio / analytics-react-native