Update to Commanders Act version 5

[defagos]

As a member of the analytics team I want results to be gathered with the new Commanders Act platform requiring version 5 of the SDKs to be running in mobile applications.

Acceptance criteria

• The Commanders Act SDK has been updated.
• Integration documentation has been updated if needed.
• The SDK is available on GitHub.
• Integrators have been notified about the change and the deadline.

Remark: Server-side tags need to be migrated as well for the whole integration to work. Handled by the GD.

Hints

• Migration to V5 is documented. In particular the events are now not anonymous data dictionaries but are specifically defined with their associated data.
• The list of available events is documented.

Tasks

• [x] Update SDK.
• [x] Gather examples to be able to discuss how each use case we support (page view, hidden event, streaming event) must be sent with v5.
• [x] Ask for source key to use.
• [x] Update instantiation.
• [x] Update request interceptor.
• [x] Update hidden event implementation.
• [x] Update API to use "event" term for general events, instead of "hidden event".
• [x] Update page view implementation.
• [x] Update stream tracking implementation.
• [x] Profile app launch time.
• [x] Update integration documentation.
• [x] Replace references to Tag Commander with Commanders Act.
• [x] Remove prod / preprod environment flag. Callers must provide source keys for debug / production instead (and pick the correct one at initialization time).
• [x] Use debug key in SRG Analytics demo.
• [x] Build and send fields which were previously built server-side, e.g. page_unique_name (if the feature is not provided by the new Commanders Act platform). All was migrated in Server side V2 in Data cleansing feature.
• [x] Generate test data for Commanders Act (Play app with v5) sent to debug source keys.
• [x] ~If SRG SSR debug / production source keys are known to be fixed provide constants for them.~
• [x] ~Related to TCCore 5.x versions, update documentation to provide information about NSBluetoothPeripheralUsageDescription and NSBluetoothAlwaysUsageDescription keys in application Info.plist. Example: "The application should not use Bluetooth. Please don’t allow.".~
• [x] Archive SRGSSR "Tag Commander" repositories.
• [x] ~Update to version 5.3.1 and figure out that a new name for the event title, event_title, conflicting with the reserved event_name from the SDK.~
• [x] Update to version 5.3.1 and use now event_name as custom event name, installed of hidden_events.
• [x] Have a look at potential related decisions.
• [x] Clarify pageType and pageName, regarding srg content_title and content_page_type properties.
• [x] Clarify rules client side and server side to split media events and events: SRGAnalytics unit tests don't sent the media_urn, which seems to be the the "media uid" for Webtrekk to group events together. Did in the migration guide.
• [x] Write a migration guide: In wiki part: migration guide.
• [x] Update migration guide with release number and remove "WIP".
• [x] Use iOS debug source key for unit tests.

[defagos]

Some questions to Commanders Act:

About the initialization

• Where can the source key be found in our case?
• Will we need one source key per app or will each BU (or maybe the entire SRG SSR) only need one key for all apps?

Answers for Commanders Act

• The source key is provided once you add a given source into the UI. E.g. if you add an SDK as a source, you will be provided with the source key in the source's settings.
• You will get your statistics and monitoring "per source key", so to say. In general, it makes sense to have one source key per account. It does depend on how you want to organise things on your end, however.

General questions

• Looking at the Android implementation it seems that any custom event parameter set might accidentally override existing official parameters like event_name, for example. This is something that was possible with version 4 of the SDK but can you confirm this is intended v5 behavior as well?
• We assume that the same answer holds for the iOS / tvOS SDK. Correct?

Answers for Commanders Act

• It is the case. Custom properties when named the same as native properties will overwrite their counterpart.
• Yes.

Page views

We have looked at TCPageViewEvent and we think it should match our needs for page view measurements, as follows:

• The page titles are sent under the content_title key with v4 and are technical non-localized page identifiers (e.g. home or player). We therefore think we should send them in page_type with v5. Is it a good idea?
• We are currently not sending any pretty page title, thus we think we should leave page_name empty. Correct?
• We are currently sending other values under other keys and we will send them to v5 in the same way. Some of them are currently processed server-side (e.g. navigation levels which are concatenated together to produce a new identifier) and we assume these rules will be migrated to the new platform as is. Correct?

Answers for Commanders Act

• Yes, I would agree with that.
• Yes, correct.
• It's not possible for now, this will be done with data cleansing and should be available as a first version near the end of the month.

Hidden events

Hidden events are any kind of other tracked events we are currently sending (e.g. the user clicked on some button related to some feature). For those we think we should use TCCustomEvent:

• The event_name should be set to hidden_event. Is it a good idea?
• We are currently sending other values but the keys we are using conflict with official Commanders Act SDK keys (e.g. event_name and event_id). For those we probably need new keys (e.g. hidden_event_name and hidden_event_id) and it is likely that the mapping to Mapp will need to be updated accordingly. Alternatively, we could use page_type to convey the event name (e.g. favorite_add for a tracked button used to add some item to user favorites) instead of introducing a custom parameter name. Which approach would be preferable?

Answers for Commanders Act

• You can name it however you want. Whatever best describes the event.
• Either would work, but the most readable way to handle this would be creating the additional properties in the payload. e.g. hidden_event_name: "favourite_add".
• Depending on what you do with these "hidden_events" and how much data is associated with them, you might want to have an entire custom event dedicated to each. E.g event_name : favorite_add. Would that make sense for you?

Streaming events

For those we think we should use TCCustomEvent:

• The kind of streaming event (e.g. play, pause, etc.) could be sent in event_name. Is it a good idea?
• The rest of the values would be sent in other parameters, as we do today.

Answers for Commanders Act

• It is a good idea. It makes sense to track these as events.
• Thats fine. Wherever we dont provide an event or parameter, a custom one can be created

[defagos]

We'll define a production / test source key pair for the SRG account, which must be supplied when configuring the tracker. These can be then retrieved from the source settings in Commanders Act web interface.

[defagos]

I had a look at our tracker configuration to see whether we could hide the source key internally (release / test), but I am not sure this is a good idea. The API is namely already complicated enough (we have prod / preprod flag, source keys are not known for some BUs yet, and we also have a unit test flag).

I am also not sure having a test source key built with SRG Analytics would be helpful (except for us during the initial validation). So adding it would only add confusion IMHO.

If we obtain one key per BU we could harcode them but otherwise we should likely still ask them at startup time. This is similar to what we were doing with the container until now anyway.

[defagos]

I had to avoid using __attribute__((constructor)) in unit tests since the new Commander Act SDK would otherwise deadlock if initialized from such a function.

[defagos]

As discussed with Commanders Act we currently filter and map some fields server-side (e.g. page_unique_name or fields which are nullable). This feature is not available yet but might be delivered next month. If missing we must be prepared to implement some logic client-side.

[defagos]

Following questions mentioned above we discussed removal of the prod / preprod environment flag. The source key can be used to pick a production or development source instead.

[defagos]

Question for @amtins : Is Letterbox web sending media_embedding_environment to Tag Commander in streaming events? If so, do you think we could drop it on the web as well?

[amtins]

@defagos neither Letterbox Web nor TP have ever sent the media_embedding_environment property to TagCommander.

[pyby]

I tried to switch to the official SPM repository with version 5.2.1. https://github.com/CommandersAct/iOSV5/releases/tag/5.2.1

TCCore is linked to <CoreBluetooth/CoreBluetooth.h>:

grep -r CoreBluetooth/CoreBluetooth.h TCCore.xcframework:

TCCore.xcframework/tvos-arm64/TCCore.framework/Headers/TCNetworkManager.h:#import <CoreBluetooth/CoreBluetooth.h>
TCCore.xcframework/tvos-arm64_x86_64-simulator/TCCore.framework/Headers/TCNetworkManager.h:#import <CoreBluetooth/CoreBluetooth.h>
TCCore.xcframework/ios-arm64_x86_64-simulator/TCCore.framework/Headers/TCNetworkManager.h:#import <CoreBluetooth/CoreBluetooth.h>
TCCore.xcframework/ios-arm64/TCCore.framework/Headers/TCNetworkManager.h:#import <CoreBluetooth/CoreBluetooth.h>

So applications needs to add NSBluetoothPeripheralUsageDescription and NSBluetoothAlwaysUsageDescription to the Info.plist to avoid "ITMS-90683: Missing purpose string in Info.plist".

But this official SPM repository has TCServerSide by default, which use IDFA and linked to AdSupport (see #51).

UPDATE: TagCommander team is working on an improvement for TCServerSide-noIDFA and SPM.

[pyby]

Update to Commanders Act 5.2.2. https://github.com/CommandersAct/iOSV5/releases/tag/5.2.2

✅ TCServerSide_noIDFA product is used. The library is not linked to AdSupport anymore. ⚠️ TCCore is still linked to <CoreBluetooth/CoreBluetooth.h>.

Checking unit tests, global default labels are now fixed: used internally with SRGIdentity logged account.

[pyby]

@defagos @StaehliJ I've updated to TG 5.3.1 versions (SPM issue again with 5.2.2), and apply the event_title renamed for hidden event "name", before to the flatten payload and the conflict with the reserved event_name from the SDK.

When change validated, we should change on both platforms (Apple and Android), maybe on the third one too, the web JS. I let you refine in your team. I could be involve if needed. https://github.com/SRGSSR/pillarbox-documentation/issues/26

[pyby]

Meeting with Julius, Mario and @StaehliJ and @waliid to clarify and update specifications:

1. PageView event
- event_name : page_view (TC predefined key)
- SDK: TCPageViewEvent
- In TC server, for MAPP Page View destination filter: event_name is page_view

2. Media events
- event_name : play, pause , stop, pos, uptime, seek, eof, segment
- SDK: TCCustomEvent
- In TC server, for MAPP Media destination filter: media_urn exists

3. Custom events
- event_name : Event name example (ScrollTo, ButtonClicked, favorite_add, …)
- SDK: TCCustomEvent
- In TC server, for MAPP Event destination filter: event_name is not page_view AND media_urn doesn't exit

For Apple SDK, here is the update:

• remove "hidden_event"
• remove "event_title"
• Set event_name as the custom event name.

EDIT: filter is now updated and share in the migration guide.

[pyby]

During a mapping review in TagCommander server V2, the content_title replaced by page_type is an issue. I fixed 2 times.

After a meeting with Julius, Mario, 2 MAPP consultants, 1 Commander Act consultant, they all spoke about page_name and page_type. Like the TC example, type and name for a page a common convention.

From documentation in confluence and SRF News iOS app code, and regarding mapping:

SRG current property	SDK TCPageView property	TC new property	mandatory
content_title	pageName	page_name	true
content_page_type	pageType	page_ype	false

For sure, type ≠ title/name We could add the page_type in the public API as pageType (String). Or ignore and set pageType to null.

Example from SRF News (UDP):

• content_page_type= "Artikel"
• content_title = "feuerwehren-kaempfen-weiter-braende- halten-griechenland-und-tuerkei-weiter-in-atem"

Example from SRF News (Webtrekk):

• content_page_type= "Landingpage"
• content_title = "uebersicht"

[pyby]

Update after a meeting with Julius and Mario:

Page views

We have looked at TCPageViewEvent and we think it should match our needs for page view measurements, as follows:

• The page titles are sent under the content_title key with v4 and are technical non-localized page identifiers (e.g. home or player). We therefore think we should send them in page_type with v5. Is it a good idea?
• We are currently not sending any pretty page title, thus we think we should leave page_name empty. Correct?
• We are currently sending other values under other keys and we will send them to v5 in the same way. Some of them are currently processed server-side (e.g. navigation levels which are concatenated together to produce a new identifier) and we assume these rules will be migrated to the new platform as is. Correct?

Answers for ADI DG team

• No. Leave null for now. But Mario will check to set is as Mandatory: As it's described in the SRG specification. maybe content_page_type property.
• No, content_title has to move to page_name. Still Mandary, String.
• All transformations do server side V1 has been be done now also server side V2.

[pyby]

Page view title is set to page_name property.

ADI team asked to set page_type as mandatory. Should we add it in the global send page view method?

[pyby]

I tried to switch to the official SPM repository with version 5.2.1. https://github.com/CommandersAct/iOSV5/releases/tag/5.2.1

TCCore is linked to <CoreBluetooth/CoreBluetooth.h>:

grep -r CoreBluetooth/CoreBluetooth.h TCCore.xcframework

[…]

So applications needs to add NSBluetoothPeripheralUsageDescription and NSBluetoothAlwaysUsageDescription to the Info.plist to avoid "ITMS-90683: Missing purpose string in Info.plist".

[…]

UPDATE: TagCommander team is working on an improvement for TCServerSide-noIDFA and SPM.

iOSV5 Commanders Act version for SPM makes dependency to TCore with a different version value.

• CA v 5.2.1: TCCore 5.2.0: https://github.com/CommandersAct/iOSV5/blob/5.2.1/TCCore/5/2/0/TCCore.zip Issue.
• CA v 5.2.2: TCCore 5.2.1: https://github.com/CommandersAct/iOSV5/blob/5.2.2/TCCore/5/2/1/TCCore.zip No more issue.

So not need anymore to add Bluetooth descriptions in info plist.