medic / care-teams

For Product Management
0 stars 0 forks source link

Data exploration for one metric #42

Closed michaelkohn closed 6 months ago

michaelkohn commented 7 months ago

Choose a metric from the dashboard and dig into the data.

latin-panda commented 7 months ago

Tasks refresh

Questions:

Contact details

Questions:

Boot time

Questions:

latin-panda commented 7 months ago

Continuation of contact details - part 1

Initial analysis in the previous comment

CHW area - contact detail page

See all the granular telemetry (includes `medic` user) [output CSV file](https://github.com/medic/care-teams/files/14369033/all_granular_contact_detail_load_CHV_Area_202402221213.csv) **SQL:** ``` SELECT metric, user_name, count, min, max, period_start FROM useview_telemetry_metrics WHERE metric LIKE 'contact_detail:d_community_health_volunteer_area:load:%' AND metric NOT LIKE '%:apdex:%' ORDER BY metric, count desc, min desc ```
Telemetry with tolerable or frustrated Apdex (`contact_detail:d_community_health_volunteer_area:load:apdex:<_>`): | user | count | min | max | date | device info | | - | - | - | - | - | - | | kendra | 3 | 13557 | 75219 | 2024-02-12 | Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs. | | beulah | 2 | 16162 | 96064 | 2024-02-12 | Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74. | | beulah | 1 | 3767 | 3767 | 2024-02-13 | Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74. | | beulah | 4 | 3188 | 10375 | 2024-02-15 | Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74. | | beulah | 1 | 3130 | 3130 | 2024-02-16 | Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74. | | jeremiah | 1 | 16164 | 16164 | 2024-02-12 | **Desktop** Mac OS X 10_15_7 Chrome 119.0.0.0 | | gilbert | 2 | 3195 | 3562 | 2024-02-14 | Android 10 Chrome 119.0.0.0 | | gilbert | 1 | 7706 | 7706 | 2024-02-15 | Android 10 Chrome 119.0.0.0 | | medic | 22 | 3038 | 4656 | 2024-02-12 | NA |
SQL: users with Apdex tolerable or frustrated **SQL:** ``` SELECT metric, user_name, count, min, max, period_start FROM useview_telemetry_metrics WHERE metric = 'contact_detail:d_community_health_volunteer_area:load:apdex:frustrated' or metric = 'contact_detail:d_community_health_volunteer_area:load:apdex:tolerable' ORDER BY metric, count desc, min desc ```

Granular level:

>> contact_data results: - _SUM(count) where max is lower than 3s_: - 55 - _Users with a max higher than 3s_: - beulah count: 4 min: 215 max: 4733 date: 2024-02-15 - jeremiah count: 1 min: 4537 max: 4537 date: 2024-02-12 <-- Desktop, prob online, ignoring for now
>> load_children (descendants of this place) results: - _SUM(count) where max is lower than 3s_: - 54 - _Users with a max higher than 3s_: - kendra count: 3 min: 3722 max: 60080 date: 2024-02-12 - beulah count: 2 min: 13884 max: 93709 date: 2024-02-12 - jeremiah count: 1 min: 7734 max: 7734 date: 2024-02-12 <-- Desktop, prob online, ignoring for now
>> load_contact_summary results: - _SUM(count) where max is lower than 3s_: - 56 - _Users with a max higher than 3s_: - none
>> load_reports results: - _SUM(count) where max is lower than 3s_: - 27 - _Users with a max higher than 3s_: - medic count: 25 min: 299 max: 3267 date: 2024-02-12 <-- Ignoring Admin - kendra count: 3 min: 2103 max: 10609 date: 2024-02-12 - jeremiah count: 1 min: 3868 max: 3868 date: 2024-02-12 <-- Desktop, prob online, ignoring for now - gilbert count: 1 min: 3461 max: 3461 date: 2024-02-15
>> load_targets results: - _SUM(count) where max is lower than 3s_: - 54 - _Users with a max higher than 3s_: - kendra count: 3 min: 3076 max: 3554 date: 2024-02-12
>> load_tasks results: - _SUM(count) where max is lower than 3s_: - 56 - _Users with a max higher than 3s_: - none
SQL: SUM(count) where max is lower than 3s: **SQL:** ``` select SUM(count) FROM useview_telemetry_metrics WHERE metric = 'contact_detail:d_community_health_volunteer_area:load:load_children' AND metric NOT LIKE '%:apdex:%' and max <= 3000 group by metric ```
SQL: Users with a max higher than 3s **SQL:** ``` SELECT metric, user_name, count, min, max, period_start FROM useview_telemetry_metrics WHERE metric = 'contact_detail:d_community_health_volunteer_area:load:load_children' AND metric NOT LIKE '%:apdex:%' and max > 3000 ORDER BY metric, count desc, min desc ```

Summary:

latin-panda commented 6 months ago

Continuation of contact details - part 2

Initial analysis CHW Area analysis

Patient - contact detail page

See all the granular telemetry (includes `medic` user) [output CSV file](https://github.com/medic/care-teams/files/14555328/patient_all_granular_contact_detail_load_202403111503.csv)
Telemetry with tolerable or frustrated Apdex (`contact_detail:f_client:load:apdex:<_>`): | user | count | min | max | date | device info | | - | - | - | - | - | - | | kendra| 4| 14605| 17466| 2024-02-16| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | kendra| 3| 16905| 20955| 2024-02-15| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | kendra| 3| 13297| 16392| 2024-02-14| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | kendra| 2 | 12338| 14965| 2024-02-17| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | debra| 1| 41608| 41608| 2024-02-12| Pixel, Android 13, Chrome 120| | beulah| 1| 22828| 22828 | 2024-02-15| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74. | | kendra | 1| 13556| 13556| 2024-02-13| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs. | | beulah| 7 | 5554| 11352| 2024-02-15| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74.| | jeremiah| 4 | 3168| 4151| 2024-02-15| Samsung, Android 11, Chrome 97 | | gilbert| 3| 5559| 6936| 2024-02-15| Android 10 Chrome 119.0.0.0| | debra| 3 | 4901| 5841| 2024-02-13| Pixel, Android 13, Chrome 120| | gilbert| 2 | 6269| 7159| 2024-02-13| Android 10 Chrome 119.0.0.0| | gilbert| 2| 5933| 8340| 2024-02-14| Android 10 Chrome 119.0.0.0| | kendra| 2 | 5568| 7007| 2024-02-13| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | debra| 2| 3547| 4282| 2024-02-14| Pixel, Android 13, Chrome 120| | kendra| 2| 3134 | 3928| 2024-02-16| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | beulah| 1 | 10392| 10392 | 2024-02-12| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74| | debra| 1| 4540| 4540| 2024-02-12| Pixel, Android 13, Chrome 120| | rodney| 1| 3623| 3623 | 2024-02-15| Android 10, Chrome 121 | | jeremiah| 1 | 3015| 3015| 2024-02-14| Samsung, Android 11, Chrome 97 |

Granular level:

>> contact_data results: - _SUM(count) where max is lower than 3s_: - 102 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | |beulah | 11 | 241 | 11201 | 2024-02-15| |kendra | 9 | 480 | 6044 | 2024-02-16| |kendra | 7 | 394 | 6346 | 2024-02-14| |debra | 6 | 102 | 3123 | 2024-02-13| |kendra | 5 | 360 | 4277 | 2024-02-13| |kendra | 4 | 588 | 6296 | 2024-02-15| |kendra | 4 | 425 | 3854 | 2024-02-17| |beulah | 2 | 364 | 3620 | 2024-02-12|
>> load_children (descendants of this contact) results: - _SUM(count) where max is lower than 3s_: - 108 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | |beulah| 11| 354| 5419| 2024-02-15| |kendra |9| 490| 8344| 2024-02-16| |kendra |7 |399| 6994| 2024-02-14| |kendra |5| 304| 6098| 2024-02-13| |kendra |4| 530| 7746| 2024-02-15| |kendra| 4| 353 |5930| 2024-02-17|
>> load_contact_summary results: - _SUM(count) where max is lower than 3s_: - 150 - _Users with a max higher than 3s_: - None
>> load_reports results: - _SUM(count) where max is lower than 3s_: - 133 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | | beulah| 11| 149| 4026| 2024-02-15 | kendra| 4| 719| 4278| 2024-02-15 | debra| 2| 486| 4752| 2024-02-12
>> load_targets results: - _SUM(count) where max is lower than 3s_: - 150 - _Users with a max higher than 3s_: - None
>> load_tasks results: - _SUM(count) where max is lower than 3s_: - 140 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | | kendra |4 |517| 3460| 2024-02-15| |kendra |4 |320 |3003 |2024-02-17| |debra |2 |877 |33910 |2024-02-12|

Summary:

Household - contact detail page

See all the granular telemetry (includes `medic` user) [output CSV file](https://github.com/medic/care-teams/files/14555344/household_all_granular_contact_detail_load_202403111504.csv)
Telemetry with tolerable or frustrated Apdex (`contact_detail:e_household:load:apdex:<_>`): | user | count | min | max | date | device info | | - | - | - | - | - | - | | beulah | 1| 144196| 144196| 2024-02-12| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74. | | kendra | 1| 44124| 44124| 2024-02-16| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs. | | lena | 1| 39474| 39474| 2024-02-13| Samsung, Android 13, Chrome 115| | debra | 1| 17718| 17718| 2024-02-12| Pixel, Android 13, Chrome 120| | jeremiah | 1| 13374| 13374| 2024-02-15|Samsung, Android 11, Chrome 97 | | beulah | 1| 12465| 12465| 2024-02-16| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74.| | kendra | 12| 3584| 10878| 2024-02-14| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | kendra | 4| 4319| 10147| 2024-02-16| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | beulah | 4| 3536| 11386| 2024-02-13| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74.| | beulah | 4| 3386| 8563| 2024-02-15| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74.| | beulah | 3| 4182 | 8679| 2024-02-16| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74.| | kendra | 3| 3033 | 5700| 2024-02-13| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | beulah | 2| 9412 | 10672| 2024-02-12| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74.| | gilbert | 2| 8035 | 8440| 2024-02-15|Android 10 Chrome 119.0.0.0 | | jeremiah | 2| 4392| 4577| 2024-02-15| Samsung, Android 11, Chrome 97| | kendra | 2| 3965 | 4154| 2024-02-17| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | jeremiah| 2| 3427| 3907| 2024-02-14| Samsung, Android 11, Chrome 97| | sammy | 1| 9146 | 9146| 2024-02-12| Android 10, Chrome 121 | | debra | 1| 6990 | 6990| 2024-02-13| Pixel, Android 13, Chrome 120 | | kendra | 1| 6429 | 6429| 2024-02-15| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | gladys | 1| 6336 | 6336| 2024-02-12| Samsung Android 9 Chrome 121| | kendra | 1| 5703 | 5703| 2024-02-12| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | gilbert | 1| 5004 | 5004| 2024-02-13| Android 10 Chrome 119.0.0.0| | debra | 1| 4641 | 4641| 2024-02-14| Pixel, Android 13, Chrome 120| | judy | 1| 4305 | 4305| 2024-02-13| Android 8, Chrome 116 | | gladys | 1| 4161 | 4161| 2024-02-18| Samsung Android 9 Chrome 121| | gilbert | 1| 3500 | 3500| 2024-02-14| Android 10 Chrome 119.0.0.0| | gladys | 1| 3179 | 3179| 2024-02-15|Samsung Android 9 Chrome 121 |

Granular level:

>> contact_data results: - _SUM(count) where max is lower than 3s_: - 140 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | | kendra| 12 | 208| 4537| 2024-02-14| | beulah| 5| 239| 4496| 2024-02-13| | kendra | 5 | 238 | 10916| 2024-02-16| | beulah | 4| 265| 3383| 2024-02-16| | beulah | 3| 301 | 10270| 2024-02-12|
>> load_children (descendants of this place) results: - _SUM(count) where max is lower than 3s_: - 136 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | | kendra | 12| 322 | 3584 | 2024-02-14| | kendra | 5 | 610 | 14548 | 2024-02-16| | beulah| 5 | 333| 3725| 2024-02-13| | beulah | 4| 390 | 5586 | 2024-02-16| | beulah| 3| 790 | 133907| 2024-02-12| | lena | 1 | 33103| 33103 | 2024-02-13| | debra | 1| 17586| 17586| 2024-02-12| | sammy| 1 | 8040| 8040| 2024-02-12| | gladys| 1 | 5483| 5483| 2024-02-12|
>> load_contact_summary results: - _SUM(count) where max is lower than 3s_: - 161 - _Users with a max higher than 3s_: - None
>> load_reports results: - _SUM(count) where max is lower than 3s_: - 135 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | | jeremiah| 17 | 132 | 4384 | 2024-02-15| | kendra | 5| 1619| 6571 | 2024-02-16| | beulah| 4| 1288| 7127 | 2024-02-15| | gilbert| 2| 1475 | 4333| 2024-02-15| | beulah| 2 | 1172| 6577| 2024-02-12| | lena | 1| 3172 | 3172 | 2024-02-13|
>> load_targets results: - _SUM(count) where max is lower than 3s_: - 163 - _Users with a max higher than 3s_: - None
>> load_tasks results: - _SUM(count) where max is lower than 3s_: - 150 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | |beulah| 5 |424| 3218| 2024-02-13| |kendra| 4| 496 |10558| 2024-02-16| |gilbert |1 |3858| 3858| 2024-02-15|

Summary:

latin-panda commented 6 months ago

Continuation of boot time

Initial analysis

See all the granular telemetry (includes `medic` user) [output CSV file](https://github.com/medic/care-teams/files/14556199/all_granular_boot_time_202403111617.csv)
Telemetry with tolerable or frustrated Apdex (`boot_time:apdex:<_>`): | user | count | min | max | date | device info | | - | - | - | - | - | - | | kendra | 2| 13886| 100357| 2024-02-12| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | beulah| 1| 119439| 119439| 2024-02-12| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74.| | jeremiah | 1| 91413| 91413| 2024-02-14| Samsung, Android 11, Chrome 97| | lena | 1 | 85327| 85327| 2024-02-13| Samsung, Android 13, Chrome 115| | judy | 1| 77579| 77579| 2024-02-13| Android 8, Chrome 116| | gladys | 1| 70746| 70746| 2024-02-12| Samsung Android 9 Chrome 121| | debra | 1| 69577| 69577| 2024-02-12 | Pixel, Android 13, Chrome 120| | sammy| 1| 62137| 62137| 2024-02-12| Android 10, Chrome 121| | gladys| 1| 55584| 55584| 2024-02-12| Samsung Android 9 Chrome 121| | jeremiah| 1| 49629| 49629| 2024-02-12| Samsung, Android 11, Chrome 97| | shelly | 1| 49225| 49225| 2024-02-12| Nokia, android 9, chrome 119| | ginger| 1 | 47451| 47451| 2024-02-14| Samsung android 11, **chrome 87**| | daryl | 1| 40313| 40313| 2024-02-12| **Desktop [Ignore]**| | beulah | 1 | 15014| 15014| 2024-02-13| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74.| | kendra| 1 | 14098| 14098| 2024-02-16| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | kendra| 1 | 14055| 14055| 2024-02-15| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | kendra | 1| 13493| 13493| 2024-02-14| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | kendra| 1| 13225| 13225| 2024-02-17| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | gilbert| 3| 6895| 7975| 2024-02-14| Android 10 Chrome 119.0.0.0| | beulah | 3| 6138| 10288| 2024-02-14| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74.| | gilbert| 2| 7869| 7938| 2024-02-13| Android 10 Chrome 119.0.0.0 | | beulah | 2| 5752| 5976| 2024-02-16| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74.| | kendra| 1| 9927| 9927| 2024-02-13| Alcatel 50560 Android 5.1.1 Webview 95.0.4638.74 , 3868 Docs = 73.70MB Docs.| | gilbert | 1| 7908| 7908| 2024-02-15| Android 10 Chrome 119.0.0.0| | beulah | 1| 6169| 6169| 2024-02-15| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74.| | beulah| 1 | 5865| 5865| 2024-02-12| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74.| | debra | 1| 3564.5 | 3564.5 | 2024-02-13| Pixel, Android 13, Chrome 120|

Granular level: See Telemetry documentation about boot time

>> boot_time:1:to_first_code_execution results: - _SUM(count) where max is lower than 3s_: - 81 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | | beulah| 3| 3765| 5995 | 2024-02-14| | gilbert| 3| 3630| 4562| 2024-02-14| | kendra| 2| 8012| 8152| 2024-02-12| | gilbert | 2| 4846| 5393| 2024-02-13| | beulah| 2| 3766| 9257| 2024-02-12| | beulah| 2| 3570| 3579| 2024-02-16| | kendra| 1| 8507| 8507| 2024-02-15| | kendra| 1| 8190| 8190| 2024-02-14| | kendra| 1| 7971| 7971| 2024-02-16| | beulah| 1| 7815| 7815| 2024-02-13| | kendra | 1| 7668| 7668| 2024-02-17| | kendra | 1| 5969| 5969| 2024-02-13| | gilbert | 1| 4462| 4467| 2024-02-15| | beulah | 1| 3787| 3787| 2024-02-15|
>> boot_time:2:to_bootstrap results: - _SUM(count) where max is lower than 3s_: - 71 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | | beulah| 3 |1933| 3437| 2024-02-14| | jeremiah| 3| 712| 89490 | 2024-02-14| | gladys| 3| 15| 55347 |2024-02-12| | kendra| 2 |4816 |90884| 2024-02-12| | beulah| 2| 1639 |109323 |2024-02-12| | sammy |2| 420| 61442 |2024-02-12| | gladys |2| 277 |70161 |2024-02-12| | deborah |2| 196 |56916| 2024-02-12| | lena |1 |84436 |84436 |2024-02-13| | judy |1| 77122 |77122 |2024-02-13| | debra |1| 68659 | 68659 |2024-02-12| | jeremiah| 1 |49562 |49562 |2024-02-12| | shelly |1| 48270 |48270 | 2024-02-12| | ginger |1| 45333 |45333 |2024-02-14| | daryl |1 |40231 |40231 |2024-02-12| | beulah |1| 4761 |4761 |2024-02-13| | kendra |1| 4463 |4463| 2024-02-16| | kendra |1| 4393 |4393| 2024-02-17| | kendra |1| 4359| 4359 |2024-02-15| | kendra |1| 4037 |4037 |2024-02-14|
>> boot_time:2_1:to_replication results: - _SUM(count) where max is lower than 3s_: - Zero - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | |beulah |1 |106942| 106942 |2024-02-12| |kendra |1| 89467 |89467| 2024-02-12| |jeremiah| 1| 89176 |89176| 2024-02-14| |lena| 1 |84219 |84219| 2024-02-13| |judy |1| 77059 |77059| 2024-02-13| |gladys |1| 70036 |70036| 2024-02-12| |debra |1| 68414 |68414| 2024-02-12| |sammy| 1| 61214 |61214| 2024-02-12| |deborah| 1| 56793| 56793 |2024-02-12| |gladys| 1| 55304| 55304| 2024-02-12| |jeremiah| 1| 49402| 49402| 2024-02-12| |shelly| 1 |48027 |48027| 2024-02-12| |ginger| 1 |45265 |45265| 2024-02-14| |daryl| 1 |40178 |40178| 2024-02-12|
>> boot_time:2_2:to_purge results: - _SUM(count) where max is lower than 3s_: - Zero - _Users with a max higher than 3s_: - None
>> boot_time:2_3:to_purge_meta results: - _SUM(count) where max is lower than 3s_: - 55 - _Users with a max higher than 3s_: - None
>> boot_time:3:to_angular_bootstrap results: - _SUM(count) where max is lower than 3s_: - 102 - _Users with a max higher than 3s_: - None
>> boot_time:4:to_db_warmed results: - _SUM(count) where max is lower than 3s_: - Zero - _Users with a max higher than 3s_: - None
>> boot_time:purging: results: - _SUM(count) where max is lower than 3s_: - Zero - _Users with a max higher than 3s_: - None
>> boot_time:purging_failed results: - _SUM(count) where max is lower than 3s_: - Zero - _Users with a max higher than 3s_: - None
>> boot_time:purging_meta: results: - _SUM(count) where max is lower than 3s_: - boot_time:purging_meta:false 43 - boot_time:purging_meta:true 55 - boot_time:purging_meta:undefined 4 - _Users with a max higher than 3s_: - None
>> boot_time:purging_meta_failed results: - _SUM(count) where max is lower than 3s_: - Zero - _Users with a max higher than 3s_: - None

Summary:

latin-panda commented 6 months ago

Continuation of task refresh

Initial analysis

See all the granular telemetry (includes `medic` user) [output CSV file](https://github.com/medic/care-teams/files/14557049/all_granular_tasks_reload_202403111726.csv)
Telemetry with tolerable or frustrated Apdex (`tasks:refresh:apdex:<_>`): | user | count | min | max | date | device info | | - | - | - | - | - | - | | debra| 2| 22752 | 29473| 2024-02-13| Pixel, Android 13, Chrome 120| | beulah| 2| 19970| 21294| 2024-02-13| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74.| | jean | 2| 17489| 18309| 2024-02-12| ???| | gilbert| 2| 16477| 17443| 2024-02-14| Android 10 Chrome 119.0.0.0| | shelly| 2| 12185| 75568 | 2024-02-16| Nokia, android 9, chrome 119| | beulah| 1 | 21733| 21733| 2024-02-15| Moto X Android 5.1, CHT Android unbranded 1.1.0, webview 95.0.4638.74. | | sammy| 1 | 13588| 13588| 2024-02-12| Android 10, Chrome 121| | daryl| 9 | 4747 | 11735| 2024-02-14| ???| | daryl| 6| 3346| 11629| 2024-02-15| ???| | daryl| 4| 3630| 11438| 2024-02-12| Desktop [Ignore] | | rodney| 3| 6372| 11702| 2024-02-15| Android 10, Chrome 121| | rodney| 3| 5899| 10211| 2024-02-13| Android 10, Chrome 121| | gilbert| 2| 11013| 11528| 2024-02-15| Android 10 Chrome 119.0.0.0| | sammy| 2| 8334| 8586| 2024-02-15| Android 10, Chrome 121| | judy | 2| 8330| 9790| 2024-02-16| Android 8, Chrome 116| | daryl| 2| 6347| 7320| 2024-02-13| ???| | debra| 2| 5371| 9734| 2024-02-14| Pixel, Android 13, Chrome 120| | debra| 1 | 11388| 11388| 2024-02-13| Pixel, Android 13, Chrome 120| | lena | 1| 11118| 11118| 2024-02-14| Samsung, Android 13, Chrome 115| | gladys| 1 | 5524| 5524| 2024-02-12| Samsung Android 9 Chrome 121| | judy| 1 | 3826| 3826| 2024-02-18| Android 8, Chrome 116| | shelly| 1 | 3805| 3805| 2024-02-14| Nokia, android 9, chrome 119|

Related Telemetry See Telemetry documentation about tasks

>> tasks:refresh results: - _SUM(count) where max is lower than 3s_: - 8 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | |daryl| 31| 1379| 171932 |2024-02-14| |sammy |30| 8334| 172893 |2024-02-15| |jeremiah| 28| 13096| 146509 |2024-02-15| |daryl| 28 |1419| 77996| 2024-02-15| |kendra| 19 |46507| 260739| 2024-02-14| |deborah| 16 |415 |45774| 2024-02-12| |jeremiah| 15 |30127| 223800| 2024-02-14| |kendra| 13| 33529 |190044 |2024-02-13| |gilbert| 12| 25801| 64656 |2024-02-14| |daryl| 11| 3630| 49800 |2024-02-12| |daryl| 11| 1372| 48728 |2024-02-13| |debra| 10| 5371| 105559 |2024-02-14| |kendra| 9| 49988| 132430 |2024-02-16| |kendra| 9| 39464| 555199 |2024-02-17| |kendra |7| 36177| 210434| 2024-02-15| |beulah |7| 12373| 107523 |2024-02-16| |rodney| 7 |5899 |25014| 2024-02-13| |judy |7| 2042| 35302| 2024-02-16| |judy |6| 44137| 79390 |2024-02-13| |lena |6| 11118| 101510| 2024-02-14| |rodney| 6| 6372| 35040| 2024-02-15| |rodney| 6 |2662 |32890 |2024-02-12| |gilbert |5 |67540167 |68161147 |2024-02-15| |shelly |5 |44742 |211729 |2024-02-12| |gilbert| 5 |38969 |79014 |2024-02-13| |gilbert |5| 11013 |520318 |2024-02-15| |gilbert |4| 58513201 |61219159 |2024-02-13| |shelly |3| 14766 |27325| 2024-02-13| |debra |3 |11388 |29473 |2024-02-13| |gilbert |3 |1144 |17443 |2024-02-14| |gladys |2 |48007783 |48909616 |2024-02-15| |beulah |2 |268661 |272635 |2024-02-14| |gladys |2 |41882 |42794 |2024-02-15| |beulah |2 |19970 |21294 |2024-02-13| |jean |2 |17489 |18309 |2024-02-12| |shelly |2 |12185 |75568 |2024-02-16| |beulah |1 |21733 |21733 |2024-02-15| |sammy |1 |13588 |13588 |2024-02-12| |gladys |1 |5524 |5524 |2024-02-12| |shelly |1 |3805 |3805 |2024-02-14|
>> tasks:load results: - _SUM(count) where max is lower than 3s_: - 56 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | |gilbert| 7| 602| 5623| 2024-02-14| |kendra| 5| 4271 |18009 |2024-02-14| |beulah |5| 3152 |54521| 2024-02-13| |jeremiah| 5| 2061| 55118| 2024-02-14| |shelly |5 |312 |13821| 2024-02-13| |daryl |5 |151| 5223| 2024-02-12| |beulah |4 |5340 |26124 |2024-02-14| |jeremiah| 4| 1795 |3734| 2024-02-15| |kendra |4| 1556| 6340| 2024-02-16| |debra |3| 6378 |12255| 2024-02-13| |gilbert| 3| 777| 14610 |2024-02-13| |kendra |2| 11167| 38985| 2024-02-12| |kendra |2 |8327| 17909| 2024-02-13| |kendra| 2| 2422| 33820| 2024-02-17| |lena |2| 1379 |3023| 2024-02-14| |shelly |2| 276 |11266 |2024-02-16| |gladys| 2| 169 |5214| 2024-02-12| |shelly |1| 43143| 43143 |2024-02-12| |gladys| 1| 32650| 32650| 2024-02-12| |beulah |1| 13885| 13885 |2024-02-12| |kendra| 1 |13696| 13696| 2024-02-15| |beulah |1| 5193| 5193| 2024-02-16| |beulah |1| 4311 |4311 |2024-02-15|
>> enketo:tasks:....:render results: - _SUM(count) where max is lower than 3s_: - enketo:tasks:household_member_registration_reminder:add:render 4 - enketo:tasks:patient_details_reminder:add:render 34 - enketo:tasks:pregnancy_home_visit:add:render 1 - enketo:tasks:referral_follow_up:add:render 81 - _Users with a max higher than 3s_: |metric |user | count | min | max | date| | - | - | - | - | - | - | | enketo:tasks:commodity_count:add:render| gilbert| 1| 12163 | 12163 | 2024-02-15| | enketo:tasks:household_member_registration_reminder:add:render| kendra| 1 | 25077| 25077 | 2024-02-13| | enketo:tasks:household_member_registration_reminder:add:render| jeremiah| 1| 7577 | 7577| 2024-02-14| | enketo:tasks:patient_details_reminder:add:render| gilbert| 7| 1063| 8047| 2024-02-14| | enketo:tasks:patient_details_reminder:add:render | beulah| 1 | 25050| 25050| 2024-02-15| | enketo:tasks:patient_details_reminder:add:render | jeremiah| 1 | 7475| 7475| 2024-02-14| | enketo:tasks:patient_details_reminder:add:render | debra | 1| 6910| 6910| 2024-02-12| | enketo:tasks:patient_details_reminder:add:render | jeremiah| 1| 5892| 5892| 2024-02-15| | enketo:tasks:patient_details_reminder:add:render| sammy | 1 | 3491| 3491 | 2024-02-12| | enketo:tasks:postnatal_care_service:add:render | kendra| 1 | 8993| 8993 | 2024-02-17| | enketo:tasks:pregnancy_home_visit:add:render| gilbert | 1| 3742 | 3742 | 2024-02-13| | enketo:tasks:referral_follow_up:add:render| jeremiah| 15| 882| 7905 | 2024-02-15| | enketo:tasks:referral_follow_up:add:render| kendra | 10| 3317| 33948| 2024-02-14| | enketo:tasks:referral_follow_up:add:render| jeremiah| 8| 1021| 8637| 2024-02-14| | enketo:tasks:referral_follow_up:add:render| beulah| 6| 2249| 7158 | 2024-02-14| | enketo:tasks:referral_follow_up:add:render| debra | 6| 914 | 33337| 2024-02-13| | enketo:tasks:referral_follow_up:add:render| kendra| 4 | 2450 | 9797 | 2024-02-16| | enketo:tasks:referral_follow_up:add:render| kendra| 3 | 20823| 40888| 2024-02-13| | enketo:tasks:referral_follow_up:add:render| debra | 3 | 1001| 7627| | 2024-02-14| | enketo:tasks:referral_follow_up:add:render| kendra| 2| 14331| 18094| 2024-02-15| | enketo:tasks:referral_follow_up:add:render| beulah| 2 | 3515| 4067| 2024-02-16| | enketo:tasks:referral_follow_up:add:render| kendra| 2 | 1958| 4774| 2024-02-17| | enketo:tasks:referral_follow_up:add:render| rodney| 2 | 1354| 6983| 2024-02-12| | enketo:tasks:referral_follow_up:add:render| lena| | 2 | 870| 3138| 2024-02-14| | enketo:tasks:referral_follow_up:add:render| shelly | 2 | 430| 5700| 2024-02-12| | enketo:tasks:referral_follow_up:add:render| kendra | 1| 27117| 27117| 2024-02-12| | enketo:tasks:referral_follow_up:add:render| debra | 1| 5958| 5958| 2024-02-15|
>> enketo:tasks:....:save results: - _SUM(count) where max is lower than 3s_: - enketo:tasks:commodity_count:add:save 1 - enketo:tasks:household_member_registration_reminder:add:save 2 - enketo:tasks:patient_details_reminder:add:save 26 - enketo:tasks:pregnancy_home_visit:add:save 1 - enketo:tasks:referral_follow_up:add:save 69 - _Users with a max higher than 3s_: |metric |user | count | min | max | date| | - | - | - | - | - | - | | enketo:tasks:household_member_registration_reminder:add:save | debra| 1 |27421 |27421| 2024-02-14| |enketo:tasks:postnatal_care_service:add:save | kendra | 1 |18441 |18441 |2024-02-17| |enketo:tasks:referral_follow_up:add:save |jeremiah |13 |437 |23241 |2024-02-15| |enketo:tasks:referral_follow_up:add:save |kendra |6 |1483 |20180 |2024-02-14| |enketo:tasks:referral_follow_up:add:save |jeremiah |6 |402 |16766 |2024-02-14| |enketo:tasks:referral_follow_up:add:save |debra |6 |363| 25131 |2024-02-13| |enketo:tasks:referral_follow_up:add:save |kendra |3 |1225 |8644 |2024-02-13| |enketo:tasks:referral_follow_up:add:save |kendra| 3 |1111 |18019 |2024-02-16| |enketo:tasks:referral_follow_up:add:save |debra |3 |328 |24392 |2024-02-14| |enketo:tasks:referral_follow_up:add:save |beulah |2 |15479 |20329 |2024-02-16| |enketo:tasks:referral_follow_up:add:save| beulah |2 |14372 |18171 |2024-02-14| |enketo:tasks:referral_follow_up:add:save |kendra |2 |2415 |5104 |2024-02-15|
>> rules-engine:tasks:all-contacts results: This is called in the Task tab, to load all tasks from households and patients. - _SUM(count) where max is lower than 3s_: - 167 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | | sammy | 35 | 323| 4850 | 2024-02-15| | jeremiah | 32 | 779 | 6629 | 2024-02-15| | kendra| 24 | 1937 | 32669 | 2024-02-14| | jeremiah | 20 | 901| 53401 | 2024-02-14| | deborah| 17 | 216 | 177402 | 2024-02-12| | daryl | 16 | 53 | 5131 | 2024-02-12| | kendra| 15 | 2165 | 12769| 2024-02-13| | kendra| 14 | 903 | 5132 | 2024-02-16| | kendra| 11 | 774 | 12166 | 2024-02-17| | debra | 11 | 534| 5466 | 2024-02-14| | beulah| 10 | 1146 | 921653 | 2024-02-16| | gilbert | 10 | 227| 12794 | 2024-02-13| | rodney| 10 | 205| 10157 | 2024-02-15| | rodney | 9 | 247 | 3098 | 2024-02-13| | kendra | 8 | 4626 | 11416 | 2024-02-15| | lena| 8 | 691 | 8270 | 2024-02-14| | shelly| 8 | 204 | 13692 | 2024-02-13| | judy| 8| 86 | 4706 | 2024-02-13| | beulah| 7 | 1576 | 18741 | 2024-02-13| | gilbert| 7 | 633 | 5724 | 2024-02-15| | beulah | 6 | 2365 | 13629 | 2024-02-14| | debra| 6 | 1734 | 16321 | 2024-02-13| | rodney| 6 | 445 | 8302 | 2024-02-12| | shelly| 6 | 349 | 42758 | 2024-02-12| | shelly| 5 | 210 | 11321 | 2024-02-16| | kendra| 3 | 6097 | 476904 | 2024-02-12| | beulah| 3 | 1724 | 911342 | 2024-02-15| | debra| 3 | 917 | 64663 | 2024-02-12| | sammy| 3 | 518| 21268 | 2024-02-12| | gladys| 3 | 76 | 5110 | 2024-02-12| | beulah| 2 | 11330 | 227093 | 2024-02-12| | gladys| 2 | 665 | 32022| 2024-02-12| | ginger| 2 | 114 | 7250 | 2024-02-14| | lena| 1 | 241693 | 241693 | 2024-02-13| | jeremiah| 1 | 5183 | 5183 | 2024-02-12|
>> rules-engine:tasks:some-contacts results: This is called to load the tasks section in the contact detail page. - _SUM(count) where max is lower than 3s_: - 275 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | |kendra| 13 |243| 10456| 2024-02-16| |beulah |5| 351 |3160 |2024-02-13| |kendra| 5| 340| 3322 |2024-02-15| |gilbert |5| 299| 3790 |2024-02-15| |debra |2| 846| 3104| 2024-02-12|
>> rules-engine:tasks-breakdown:some-contacts results: This is called in task group. - _SUM(count) where max is lower than 3s_: - 131 - _Users with a max higher than 3s_: |user | count | min | max | date| | - | - | - | - | - | | kendra | 6 | 712 | 4747 | 2024-02-14| | kendra | 3 | 1225 | 4600 | 2024-02-17| | kendra | 2 | 884 | 3722 | 2024-02-15| | beulah | 2 | 663 | 3175 | 2024-02-14| | lena | 2 | 193 | 5538 | 2024-02-14|

Other telemetry not explored:

Summary:

latin-panda commented 6 months ago

Summary

This research is based on data collected from February 12th to February 17th, 2024. We invited 17 participants, of which 2 had low-end phones (small RAM, Android 5, and Chrome 90). Each had assigned an offline CHW with 3000 to 3800 documents.

This test party was based on the eCHIS Kenya config. Other configs can produce different Apdex scores, therefore highlighting different problems.

Contact details

Our test-party participants accessed three contact details pages: CHW Area, household, and patient. Those with low-end phones experienced poor performance (Apdex: Tolerable and Frustrated), while the others had primarily satisfactory performance.

CHW Area When accessing the CHW Area with approximately 110 households, the two users with low-end phones experienced extreme slowness when loading descendants (households). This was recorded 5 times, ranging from 3 seconds to 1.5 minutes. One user with a low-end phone experiences 10s loading reports for his CHW Area.

Households Eight users experienced non-satisfactory performance when accessing the households (each has 5 to 8 patients). The two users with low-end phones experienced slowness when loading the contact data (household). This was recorded 4 times, ranging from 4 seconds to 11 seconds. The eight users had poor experience when loading descendants (patients). This was recorded 7 times, ranging from 5 seconds to 2.2 minutes. The user taking 2.2min was using a low end phone. Five users had a tolerable experience when loading reports, which was recorded a total of 5 times with a duration of 4s to 7s. Lastly, one user with a low-end phone experienced 1 time duration of 10s when loading tasks.

Patient When accessing a patient page, the two users on a low-end phone experienced slowness when loading contact data. This was recorded 5 times, ranging from 4 seconds to 11 seconds. Similarly, when loading descendants it was recorded 6 times with a duration ranging from 5 seconds to 8 seconds. These two users and one user with a higher-end phone experienced 3 times a duration of 4s when loading reports. While one user with a higher-end phone experienced 1 time (first day of the test party) a duration of 33s when loading tasks.

P.d. CHT can technically support descendants under "patient"; however, for this config, the patient doesn't have descendants. The load descendants process probably didn't return anything for the patient.

Boot time

The test party participants accessed the app using their phones for the first time on February 12th and 13th. During that time, most of the non-satisfactory experiences were observed.

Two users using low-end phones had a tolerable experience during the first code execution (boot_time:1:to_first_code_execution), which was recorded 12 times and had a duration ranging from 4s to 9s.

Next is the bootstrap process (boot_time:2:to_bootstrap); this process is when purging, initial replication, and other executions happen. Eleven users recorded 19 times a duration from 4s to 1.8min; from these, 8 took more than 1min. This is probably related to the first-time access because 12 users registered an initial replication (boot_time:2_1:to_replication) from 4s to 1.7min. From those, 9 took longer than 1min.

P.d. The initial replication was improved significantly last year or so.

Tasks refresh

Tasks refresh was harder to explore just by looking at the data. When visiting the Tasks tab, one process gets the list of tasks, which is recorded differently; the first time the Task tab loads, it records task-load with 3 occurrences of +40s duration. The same process executes again when new data is synced into the phone, a task is submitted, etc. This is recorded as task-refresh; even though it is the same process as task_load, the duration recorded was quite different. There are 26 records of +40s, many of which took several minutes and 2 of which took hours.

The granular telemetry we can use to explore this behavior more is rules-engine:tasks:all-contacts. The rules engine is in charge of getting the list of tasks of all contacts that is eventually displayed on the tasks list page. The beulah user has a low-end phone, in 2 different days, it recorded 15min duration on rules-engine:tasks:all-contacts. This slowness doesn't seem to be related to the number of dirty contacts because only 2 were recorded (rules-engine:tasks: dirty contacts); this is not conclusive yet; we still need to do a deeper investigation on the data associated with these 2 dirty contacts and other processes around the calculation of tasks.

Conclusion

It's unsurprising that low-end phones have a poorer experience due to their low memory, processor, and storage capability. However, I would still have liked to see data from more low-end phone users during this test-party and observe the variation in their data. Keeping higher-end phones in the test activities is fine, we cannot tolerate low performance in hardware with good resources.

For this test-party, we generated a large number of similar tasks, which might not represent reality. I suggest taking production data from 3 different partners and looking at the task-refresh, task-load, and rules-engine:tasks:all-contacts to see if they are indeed having performance times greater than 3s. These three metrics are already being recorded.

Basing only in the test-party data and If I had to pick today some areas to analyze deeper in the code to find performance improvements, I would pick:

The initial replication was greatly improved recently; perhaps some improvements can be made in terms of scoping information to users better so that they download only what they need—like discarding adjacent places that don't provide services.

latin-panda commented 6 months ago

@michaelkohn summary is ready and closing this as completed.