medic / care-teams

For Product Management
0 stars 0 forks source link

tasks:refresh apdex: Establish baseline #117

Open michaelkohn opened 2 months ago

michaelkohn commented 2 months ago

Perform the baseline test execution to capture telemetry data for tasks:refresh

ralfudx commented 2 weeks ago

Test execution has been completed using Safaricom Neon Ray Ultra S physical device on this instance - https://care-teams-4x.dev.medicmobile.org/ and this user - gladys

Test Sequence:

  1. Run test suite (9x10 tests) - June 12 - Augustineville's Household

    • Sync data - June 13 [90 Passing]
  2. Run test suite (9x10 tests) - June 12 - Sammiemouth's Household

    • Sync data - June 13 [90 Passing]
  3. Run test suite (9x10 tests) - June 12 - Bothell's Household

    • Sync data - June 13 [89 Passing, 1 Failing]
  4. Run test suite (9x10 tests) - June 13 - Bergefort's Household

    • Sync data - June 14 [90 Passing]
  5. Run test suite (9x10 tests) - June 13 - Idellaside's Household

    • Sync data - June 14 [90 Passing]

Next step is to analyze the data and confirm that we have telemetry captured for tasks:refresh

ralfudx commented 2 weeks ago

Test execution images

Screenshot 2024-06-13 at 01 28 57 Screenshot 2024-06-13 at 08 35 42 Screenshot 2024-06-13 at 10 22 43 Screenshot 2024-06-13 at 11 47 27 Screenshot 2024-06-13 at 13 20 37
ralfudx commented 2 weeks ago

While running the tests, at some point the reports page had an unexpected behaviour where instead of showing No more reports after displaying the reports' list, it showed No reports found. - This is why we have 1 failing test in the 3rd test execution above

Screenshot_20240612-223836

ralfudx commented 1 week ago

Kenya Docs: 1757 Offline user - CHW (CHP) Safaricom Neon Ray phone (2GB RAM) Automation test (generic settings file)

Screenshot 2024-06-21 at 23 44 16
latin-panda commented 5 days ago

Telemetry file for future reference kenya-jun-20-task-refresh.json

Highlight of task:refresh and task:list in milliseconds:

Screenshot 2024-06-20 at 4 58 13 PM

Apdex of other pages:

Screenshot 2024-06-25 at 3 48 22 PM
ralfudx commented 4 days ago

The test re-run has been completed. please see details below:

Kenya Docs: 1757 Offline user - CHW (CHP) gladys Safaricom Neon Ray phone (2GB RAM)

Test Execution Screenshot 2024-06-26 at 07 50 00
Test Sequence - Run test suite (20x5 tests) -> June 25 - Bergefort's Household - Sync data -> June 26 [100 Passing] - `"aggregate_date": "2024-06-26T06:51:20.967Z"`

Telemetry data