search

mozilla-mobile / telemetry-ios

A generic library for sending telemetry pings from iOS applications to Mozilla's telemetry service.
Mozilla Public License 2.0
28 stars 23 forks source link

When dailyUploadCount is exhausted, only one ping per day is uploaded and pings in between are dropped #95

Closed st3fan closed 6 years ago

st3fan commented 6 years ago

This is related to #94 (dailyUploadCount is never reset) or possibly the same bug.

First I have exhausted the dailyUploadCount for both core and mobile-event pings.

2018/04/18 19:13:42 200 E54F00C4-C146-4824-B47C-B386495BF6FF core           D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 2018-04-18           96 (23h13m42.177890806s)
2018/04/18 19:13:42 200 D15F9376-3EBE-4FE0-9DDC-234CBDA0E436 core           D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 2018-04-18           95 (23h13m42.186699519s)
2018/04/18 19:13:42 200 F9F70C3B-2292-40F7-8EF7-462EFDA1527B core           D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 2018-04-18           85 (23h13m42.199576837s)
2018/04/18 19:13:42 200 C4C7E0C5-2299-496B-BBEE-F6C5C74E17E6 core           D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 2018-04-18           76 (23h13m42.209898593s)
2018/04/18 19:13:44 200 0C9D6EBF-FEC8-4501-804F-76C185A6E579 core           D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 2018-04-18           97 (23h13m44.903170668s)
2018/04/18 19:13:47 200 C2FEA081-BF5F-4A99-8217-732368871683 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093225819        47 (2.101412539s)
2018/04/18 19:13:47 200 AA52D27C-B1E6-43DB-8084-B070F90EBCDA core           D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 2018-04-18           98 (23h13m47.103732801s)
2018/04/18 19:13:49 200 37B98382-D44D-49EA-95D0-138A3CF99C5C core           D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 2018-04-18           99 (23h13m49.395958858s)
2018/04/18 19:13:50 200 B1081C57-8D6B-45AA-8FB0-D6CD0E56EB74 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093229966        48 (1.867367179s)
2018/04/18 19:13:50 200 469BB137-3DFA-46DB-965E-869F75EB78CA core           D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 2018-04-18           100 (23h13m50.870129075s)
2018/04/18 19:13:51 200 56130A5B-DFAF-4B93-88E0-4F7EEC638F7A core           D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 2018-04-18           101 (23h13m51.188331735s)
2018/04/18 19:13:54 200 A8393BE1-5CDA-44B5-8AD2-186A118B0FB3 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093233707        49 (1.592680481s)
2018/04/18 19:13:56 200 EA5F8128-4EB9-42E8-A6CF-96E7389F9DFC mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093236096        50 (430.42869ms)
2018/04/18 19:13:59 200 E2C98CD4-81EF-4020-87F1-AB6563977979 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093238434        51 (1.358317158s)
2018/04/18 19:14:01 200 E808B8B0-EC0F-4A5C-8AAE-AA1D18BB1A73 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093240857        52 (1.192624921s)
2018/04/18 19:14:05 200 0EAA1A9E-6997-45C7-AFB8-3B811894ADE7 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093243951        53 (2.223222611s)
2018/04/18 19:14:07 200 4FA49DCF-2CAF-40ED-9F6E-CF248B4C8006 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093246753        54 (1.08567929s)
2018/04/18 19:14:10 200 42DB9F01-0601-464A-80B9-73E3A016CF6D mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093249774        55 (1.715481344s)
2018/04/18 19:14:14 200 BF9A8059-A312-4896-A171-24774D1555F2 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093253102        56 (1.058580818s)
2018/04/18 19:14:21 200 680FCCC7-37DB-46B3-95EF-1C7A83FD01BB mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093260498        57 (1.470411712s)
2018/04/18 19:14:23 200 F276DBF7-E21C-459D-8D05-110E37D20EBD mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093263049        58 (386.625983ms)
2018/04/18 19:14:26 200 6D04BA1A-AD0F-4C72-8B1B-C76A2527BD83 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093265491        59 (1.394916838s)
2018/04/18 19:14:28 200 08E7BF38-6548-4657-810B-F83FB90AFF45 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093267858        60 (1.194921305s)
2018/04/18 19:14:30 200 B286CF66-32C7-4399-A76C-1F2F70956C8C mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093269712        61 (1.467547289s)
2018/04/18 19:14:32 200 4B6F65ED-DC5C-479F-A713-CFD0AE6E785F mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093271790        62 (1.124002614s)
2018/04/18 19:14:34 200 3FC5497E-C460-4546-B251-1BF13F5087F7 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093273876        63 (1.777316672s)
2018/04/18 19:14:36 200 D13B9CE7-1AC0-42CD-8F4B-5A0ABD367E9B mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093276374        64 (711.537302ms)
2018/04/18 19:14:39 200 C5085CE5-C156-4750-903A-A723121BC07E mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093279009        65 (918.701543ms)
2018/04/18 19:14:45 200 46D4652F-3FA1-4638-9CCC-7589411FF588 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093284357        66 (1.647379234s)
2018/04/18 19:14:47 200 38B52F14-B90F-462E-99BC-8748AB1BD02F mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093287178        67 (513.538277ms)
2018/04/18 19:15:01 200 EB8CC84F-8903-446B-BF70-FB12AF6D9738 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093300037        68 (1.820533398s)
2018/04/18 19:15:06 200 29B873EC-3045-465C-AEE6-3573D8B03A37 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093304657        69 (2.76069043s)
2018/04/18 19:15:10 200 B341F38F-DAE6-41D0-8BD0-C875A16BA493 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093309105        70 (1.588811568s)
2018/04/18 19:15:15 200 577D262E-1A30-407C-9D9B-420D40D2A6A3 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093313898        71 (2.324274708s)
2018/04/18 19:15:19 200 4AEE712F-C3E6-44CC-82EB-D9A273AD4793 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093317717        72 (2.224407883s)
2018/04/18 19:16:17 200 7AE75C4F-4F88-424F-AAEA-F2C9276C205F mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093376735        73 (1.070051309s)
2018/04/18 19:16:20 200 E228BF5A-271A-4EEC-AE14-D34D726FC6AB mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093379472        74 (1.249859816s)
2018/04/18 19:16:21 200 9D8A18D8-C51D-4BE9-AC3F-FC7714CDB9CB mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093381601        75 (932.68217ms)
2018/04/18 19:16:24 200 A5F74694-7EFE-4AD0-A64F-AEDEA890004D mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093383777        76 (1.519696832s)
2018/04/18 19:16:26 200 C0633C2D-0E88-4B93-9311-C969055F9023 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093385970        77 (1.304999262s)
2018/04/18 19:16:28 200 CE149394-642A-4AF1-BC6B-F6925C82E5BA mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093388120        78 (926.197271ms)
2018/04/18 19:16:30 200 382E89A2-C728-438D-9A75-8F83E7ADA1D5 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093390294        79 (630.015719ms)
2018/04/18 19:16:33 200 8D6F43F7-6760-4451-AD11-1EFBCB27EE67 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093392339        80 (1.115967758s)
2018/04/18 19:16:35 200 EDF24357-2130-41E4-86D7-414EC8D0C985 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093394688        81 (1.027845742s)
2018/04/18 19:16:37 200 B0EDF762-59F6-41CC-84A4-351E044DAE02 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093396658        82 (1.395123669s)
2018/04/18 19:16:39 200 B49E906A-B5D1-4C88-895F-7F18EC043FAA mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093398839        83 (1.173969916s)
2018/04/18 19:16:41 200 3E116E8A-361B-4565-A638-78FFF27D01E5 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093400909        84 (1.667627398s)
2018/04/18 19:16:43 200 0FA3147A-DCC2-4A13-8220-51FB315BA389 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093403099        85 (431.811459ms)
2018/04/18 19:16:45 200 5BD77442-FDCB-4268-939D-EE44333D8C3D mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093405012        86 (722.520052ms)
2018/04/18 19:16:47 200 BF03E385-F0A7-4F45-97A5-9DCBFF7AC360 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093407081        87 (414.373008ms)
2018/04/18 19:16:49 200 454A74B1-91EA-4244-AC7F-61F8462608B7 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093409249        88 (972.626688ms)
2018/04/18 19:16:51 200 C4588B75-01C1-4437-82BB-FE1A8A929AA4 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093411330        89 (667.225124ms)
2018/04/18 19:16:54 200 9DEE0EE0-440F-4822-A7A7-4662EEE66431 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093413257        90 (1.044614409s)
2018/04/18 19:16:55 200 AA77D988-F608-4A80-BE6A-B15B1594ED63 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093415368        91 (704.134932ms)
2018/04/18 19:16:58 200 7D0E6BA8-D3F7-4F76-B671-BCA988A64758 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093417385        92 (1.095810719s)
2018/04/18 19:16:59 200 78B9E73C-D5EB-4842-9D0E-E8991AFBF45E mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093419376        93 (709.305637ms)
2018/04/18 19:17:02 200 013C46E1-031F-4B2B-9314-938B5D993EEC mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093421306        94 (1.031018671s)
2018/04/18 19:17:03 200 D33FBB10-638A-446B-A6AB-03DD19D81CB5 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093423520        95 (856.909732ms)
2018/04/18 19:17:06 200 639C9DDD-FC11-4475-BB3A-3199F327659E mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093425606        96 (1.326465268s)
2018/04/18 19:17:07 200 AF8A41C1-F613-412E-B224-94EE28213AB6 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093427542        97 (878.964055ms)
2018/04/18 19:17:10 200 BED4ED54-1E91-49DC-B2C9-881E7BE5B4AA mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093429623        98 (1.356389906s)
2018/04/18 19:17:12 200 03354C23-83BF-4832-A606-AA03C59EDA13 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093431674        99 (1.007723048s)
2018/04/18 19:17:14 200 CD15249B-63C6-4CF7-BA52-9648507D83BE mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524093433576        100 (1.213981335s)

At this point no pings are sent anymore.

I backgrounded/foreground the application a few more times to verify this. I don't have access to the filesystem but I assume these events are now cached / queued. Is this actually true, we should pull the files from the device and check if they are actually queued

To cheat a little, I changed my iPhone's date to tomorrow.

Activating Firefox now shows the following ping upload activity:

2018/04/18 20:01:29 200 6BA47D28-2F7F-4369-A179-7D6613901372 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524096052006        106 (37.308303187s)
2018/04/18 20:01:29 200 4B31F4C8-2CD8-44EA-96C7-4E01C1D4A304 core           D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 2018-04-18           155 (24h1m29.310057822s)

Only one ping is sent, leaving gaps between the last one sent and the last one generated. The sequence numbers went up, but those pings are missing. Or at least not sent.

When I backgrounded/foreground the application a few more times, I do not see any events being sent. This is incorrect because the day changed and we should have another dailyUploadCount budget.

Again I change my iPhone's date to a later date. Activating Firefox now shows this activity:

2018/04/18 20:20:28 200 80036949-9F2C-4B08-953F-547EDA03A1B7 core           D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 2018-04-18           195 (24h20m28.674253391s)
2018/04/18 20:20:28 200 4D1F6C43-6454-49BF-9F71-C4442EBF98B6 mobile-event   D2F01E77-9AE2-4470-BE33-7FE34E3A5C3F 1524190802101        111 (-25h59m33.324809368s)

Again just the last event generated is being uploaded. The events in between are lost or ignored. Worse, this event is not from the first attempt. There are not two major gaps of events never being sent.

st3fan commented 6 years ago

What is really interesting is that even though the last events uploaded have sequence numbers 194 and 111, I see the following in the org.mozilla.ios.Fennec.plist (Preferences):

telemetry-key-prefix-core-seq: 275
telemetry-key-prefix-mobile-event-seq: 113

And I see -t-$TIMESTAMP.json files with sequences going up to those numbers.

justindarc commented 6 years ago

Closing as a duplicate of Issue #94