Investigate performance issues affecting WDF

[Rkareko]

Describe the issue to be researched Investigate way to fix performance issues reported following UAT done for WDF

Context The following is the resource breakdown for the data used for the UAT Total: 129037 Resources Patients: 1927 Resources QuestionnaireResponse: 1927 Observation: 29977 Flag: 15377 Condition: 2562 Appointment: 1740 Task: 542 Encounter: 74985

Phone details i.e make, model, Android OS version and RAM We have the following two models

• Samsung Galaxy A05 (used for UAT) ONE UI 6.1 OS: Android 14 RAM: 4GB
• Samsung Galaxy A04

Issues reported (by the client)

1. Initial sync
   • Reported to have taken a couple of hours
   • For the initial first time sync the device screen should always be on. In case if the screen goes off, some resources are not synced at all
2. QR code search
   • The query in the register took about 1 min
   • If a query had a match, the "automatic opening" of the patient record took as long as 5 minutes
3. Register search
   • Search by QR code (short code)
     • Sometimes it took around 8 - 10 mins and 0 Results
     • Sometimes few seconds delayed to start the search.
   • Search by Name
     • Randomly works
   • Search by PHN (ID)
     • 6 - 8 Mins
     • Sometimes (rarely) 3 -5 seconds to retrieve
   • Overall
     • Sometimes does not return any results or un-relevant results. Example: when I search for 8979, the PHN (ID) / QR Code / Phone does not have 8979 as part of it. But still it returns one record.

• Based on Android studio profiler (when performing a search) - This is from the client
  • CPU usage not more than 60%
  • Memory usage not more 500MB

[Rkareko]

@Mstjamush Test data is needed for validation of the reported issues.

[Mstjamush]

@Rkareko Please provide the following

1. payload metadata
2. Payload for creating a client/patient
3. payload for creating a task
4. Payload for creating a condition
5. payload for creating Questionnaires and QuestionnaireResponse
6. Payload for creating Encounters

[Rkareko]

Performance improvements when loading a profile. We have identified potential fixes for this

• Removing nested related resources especially Encounters. The filtering for these would be done using rules. This has been implemented in the profile configs and has greatly improved performance.
• Applying sorting at the db level on the related resources instead of using rules
• Removing unused rules

[Rkareko]

Performance improvements for search. The proposed solution is to move the search from the current implementation that uses rules to search against data loaded into the register paging source. The search would be moved to the db level. The use of dynamic queries significantly improves the search time. The limitation here is that the SDK does not provide a way to specify complex operations e.g search for Patients that are active AND ( Patients whose name contains the search term OR Patients whose identifier is equal to the search term). There is a requirement for the search bar to support search by either patient name or patient identifier (QR code, PHN e.t.c) The following are proposed solutions

1. Enhance the SDK search functionality to support complex search operations such as Patients that are active AND ( Patients whose name contains the search term OR Patients whose identifier is equal to the search term). A Call has been schedule to discuss this with the SDK team on Tuesday 22nd October.

   Pros

   • Appears to be the more scalable option

   Cons

   • Due to the intricate nature of the search functionality the LOE for this will likely be higher

2. Implement a custom search param that will extract all the required items i.e Patient.name, Patient.identifier into the StringIndexEntity table. Then define a data query that will search for either the patient name or patient identifier in the newly defined custom search parameter (patient for this instance) The following image shows how the items would be indexed. The index name used is patient (A more appropriate name is required)

The data query would be as follows

"dataQueries": [
          {
            "paramName": "patient",
            "filterCriteria": [
              {
                "dataType": "STRING",
                "modifier": "CONTAINS",
                "dataFilterLinkId": "name-search-query"
              }
            ]
          }
        ]

Pros

• Considerably lower LOE since the changes are on FHIRCore

Cons

• Performance concerns for search when data grows since we are introduction new data into the StringIndexEntity table
• Scalability of the solution in terms of adding new search params. Part of this may be adressed by enhanceing the process of adding custom search parameters to be configurable

[dubdabasoduba]

@Rkareko Based on the project timelines, I would be okay with option 2 since it has less LOE.

@pld we will need time to update this to use Option 1 which is more scalable

[faith-mutua]

@Rkareko I have made the following observations when doing performance testing on the qa apk

1. How long QR code search and opening the patient profile - 52.45s
2. How long QR code search takes - 5s
3. How long name search takes - 5s
4. How long PHN search takes - 8.5s
5. How long opening a patient profile takes - 33.3s

[pld]

@Rkareko Based on the project timelines, I would be okay with option 2 since it has less LOE.

@pld we will need time to update this to use Option 1 which is more scalable

I am happy with that

[faith-mutua]

Apk Version - opensrp-dc-fhir-qa-performance_improvements.apk Number of patients - 7783

@Rkareko I have made the following observations today

1. Searching by name - 1 min 12 seconds I searched using the name Ananya which returned 19 patients with that name
2. Subsequent search by name - 55.10s I searched using the name Swara which returned 73 patients with that name
3. Searching by PHN - 34.5s
4. Searching by QR code until the opening of patient profile - 22s Subsequent Searching by QR code until the opening of patient profile - 15.52s
5. Opening a patient profile - 12.68s