Not enough reports for P3A NTP-SI total count metric

[DJAndries]

We are receiving more NTP-SI view reports compared to total count reports. For all other platforms the number of total count reports exceeds the view reports.

It may be the case that the total count metric is not being reported everyday, regardless of if the user viewed a campaign.

[GeetaSarvadnya]

@DJAndries @kylehickinson I am not sure which NTP SI metrics you are referring here, please do mention the metrics names in the issues.

I think the metrics implemented in this issue is Brave.NTP.SponsoredNewTabsCreated.2 as per the issues description. correct me if I am wrong. Thanks!

[GeetaSarvadnya]

Verification INPROGRESS on Brave Nightly version 1.65.77 running iPhone 8 - iOS version 16.7.5

Verified the test plan from https://github.com/brave/brave-ios/pull/8676 Prerequisites: Open Settings -> BraveCoreSwitches -> Enable the switch Express Rotation Interval, enter the 10 in the rotation interval text box and click on Save

Steps folllowed:

1. Clean profile 1.65.77, follow prerequisites step mentioned above
2. View NTP Sponsored images
3. Open Setting via app menu
4. Click on View Chromium Local State file
5. Search for the metrics Brave.NTP.SponsoredNewTabsCreated.2
6. The metrics is not shown in local state
7. Open View Brave Histogram (p3a) Logs via settings app menu
8. Search for the metrics Brave.NTP.SponsoredNewTabsCreated.2
9. The metrics is shown

Local state	Histogram

@kylehickinson: The metrics Brave.NTP.SponsoredNewTabsCreated.2 is not shown on the Chromium Local State file even after viewing 2 to 3 NTP Sponsored image tabs. But the metrics is shown under View Brave Histogram (p3a) Logs. Is this expected? To verify the different response values mentioned via wiki (shown in image below) we need to show the values under local state file. Is there any other way to verify the different response values. Let me know your thoughts on this?

[kylehickinson]

Hi @GeetaSarvadnya, this particular issue is not the bucket for Brave.NTP.SponsoredNewTabsCreated, it's for the dynamic NTP-SI metrics, sorry for the confusion. You'd be looking to make sure those dynamic p3a metrics are being updated. Those are the ones that start with creativeInstanceId

[GeetaSarvadnya]

@kylehickinson Verified the issue again. After viewing NTP SI, I am seeing creativeInstanceId under "dynamic_metrics" via local state file. But there is some difference between the desktop and iOS creativeInstanceId metrics value. In Desktop the metrics value is shown as "creativeInstanceId.d27f16ec-a9bf-4e77-aaa7-bd778ccf482e.views": 2 but in iOS the metrics value is "creativeInstanceId.total.count": 2. Is this expected?

Desktop	iOS

[kylehickinson]

@GeetaSarvadnya I believe both creativeInstanceId.total.count and creativeInstanceId.{uuid}.{views/clicks} are expected. I suspect we may be seeing different things based on the p3a debug flags that are added, maybe @DJAndries can clarify. You could also verify in the histograms viewer instead of local state

[GeetaSarvadnya]

@kylehickinson This is what I see on Histograms (p3a)

[kylehickinson]

Seeing zero there means you didn't view any sponsored images within the p3a rotation

[GeetaSarvadnya]

@kylehickinson: In Histograms (p3a) the metrics value is always 0 even after viewing 4 to 6 NTP SI

[kylehickinson]

Here's what I see on 1.65.73 after viewing a NTP and clicking on its logo:

[GeetaSarvadnya]

@kylehickinson The creativeInstanceId.total.count value will always be 0 even when I view n number of NTP SI's, this value turns 1 only when I click on NTP SI logo

Steps that I have followed:

1. Clean profile 1.65.77
2. Open Settings -> BraveCoreSwitches -> Enable the switch Express Rotation Interval, enter the 30 in the rotation interval text box and click on Save
3. Open Settings -> BraveCoreSwitches ->Enable Don't randomize upload intervals
4. Open few NTP and view 2 NTP SI
5. Open Histogram (p3a)
6. I see creativeInstanceId.total.count =0 https://github.com/brave/brave-ios/issues/8673#issuecomment-1994793010
7. Open few NTP and click on logos of 2 NTP sponsored images
8. Open Histogram (p3a)
9. I see creativeInstanceId.total.count =1 Also, click and land events recorded but there is no view event

step 9	step 9

[kylehickinson]

Thanks I'll double check the view events, maybe there is something odd going on with those. I think that's out of scope with this ticket in particular though which simply ensures that things are reported even when you don't visit NTPs, we can open a new ticket for the view events not showing up without clicking though

[GeetaSarvadnya]

Verification PASSED on Brave Nightly version 1.65.77 running iPhone 8 - iOS version 16.7.5

• Verification notes are available in https://github.com/brave/brave-ios/issues/8673#issuecomment-1995058230
• Confirmed that the creativeInstanceId.total.count value is 1 on Histogram (p3a) page, but the value 1 is shown only when the NTP SI logo is clicked, which is incorrect, the value should 1 even for viewing NTP SI. Hence logged the follwing issues.

Logged 2 follow up issues:

https://github.com/brave/brave-ios/issues/8794 https://github.com/brave/brave-browser/issues/36794