[SR Card 2] Offline report cards corrections & optimisation

nupoorkhandelwal commented 4 months ago

Context

Currently, there are two issues with Offline reports cards available for SR -

Slowness
Incorrect numbers, cards do not check the latest values of HB or BMI to show count

Update logic for the following cards so that latest value is picked to check the criteria

[x] Adolescents having severe malnutrition
[x] Adolescents with severe anaemia
[x] Adolescents having moderate anaemia
[ ] Test once for other cards as well, if same issue is happening

Performance improvement / Slowness

Referring this readme link, update queries accordingly
Note down which card takes how much time on Prod and dev environments, then post optimisation check the timings again. Add details in the comments.

adamsanadi6 commented 4 months ago

The logic is correct, it considers only the latest value.

Time taken to run the report:

Adolescents having severe malnutrition - 16 ms(uat), 2903 ms(prod)
Adolescents with severe anaemia - 18 ms(uat), 1925 ms(prod)
Adolescents having moderate anaemia - 22 ms(uat), 1929 ms(prod)

Changed the logic for below:

Adolescents who dropped out of school - 7 ms(old logic was taking 16 ms)(uat), 3444 ms(prod)

adamsanadi6 commented 4 months ago

Need to pair on this with @Gojo-Taqi .

Gojo-Taqi commented 4 months ago

Backup queries-

Adolescents having severe malnutrition

'use strict';
({params, imports}) => {
return params.db.objects('ProgramEncounter')
    .filtered(`programEnrolment.individual.voided = false AND programEnrolment.voided = false AND programEnrolment.program.name = 'Adolescent' AND programEnrolment.programExitDateTime = null AND voided = false AND encounterDateTime <> null AND (encounterType.name = 'Annual Visit - Baseline' OR encounterType.name = 'Annual Visit - Endline')`)
    .filtered('TRUEPREDICATE sort(programEnrolment.individual.uuid asc , encounterDateTime desc) Distinct(programEnrolment.individual.uuid)')
    .filter(enc => enc.getObservationReadableValue('BMI (kg/m²)') <= 14.5 )
    .map(enc => enc.programEnrolment.individual)
};

Gojo-Taqi commented 3 months ago

Pre-release stats

Adol having sever malnutrition - 11439, 11393 (Updated - 21986,14640,15866)

Below is the script that was used after referring to the readme

'use strict';
({params, imports}) => {
    const isChildUndernourished = (enrolment) => {
      const encounter = enrolment.lastFulfilledEncounter('Annual Visit - Baseline', 'Annual Visit - Endline'); 
       if(_.isNil(encounter)) return false; 

//      const obs = encounter.findObservation("BMI (kg/m²)");
      return encounter.findObservation("BMI (kg/m²)").getValue() <= 14.5;
    };

    return params.db.objects('Individual')
        .filtered(`voided = false and SUBQUERY(enrolments, $enrolment,$enrolment.program.name = 'Adolescent' and $enrolment.programExitDateTime = null and $enrolment.voided = false).@count > 0`)
        .filter((individual) => _.some(individual.enrolments, enrolment => enrolment.program.name === 'Adolescent' && _.isNil(enrolment.programExitDateTime) && !enrolment.voided && isChildUndernourished(enrolment)))
};

Gojo-Taqi commented 3 months ago

Moving this card back to support ready

Gojo-Taqi commented 3 months ago

Link to the support ticket - https://avni.freshdesk.com/a/tickets/3655

BEULAHEVANJALIN commented 3 months ago

.filter(enc => enc.getObservationReadableValue('BMI (kg/m²)') <= 14.5) is the one slows down the report as it extracts the observation value and then checks if it's less than 14.5. And we can't do this check at the realm level as observations.valueJSON is of type JSON string, and it's impossible to parse JSON strings directly within Realm queries. So the most optimal script we can do is:

'use strict';
({params, imports}) => {
    const bmiObservationUUID = '97e3a847-f41b-42e2-beaf-ad6c22bdc227';
    return params.db.objects('ProgramEncounter')
        .filtered(`programEnrolment.individual.voided = false AND 
                    programEnrolment.voided = false AND 
                    programEnrolment.program.name = 'Adolescent' AND 
                    programEnrolment.programExitDateTime = null AND 
                    voided = false AND 
                    encounterDateTime <> null AND 
                    (encounterType.name = 'Annual Visit - Baseline' OR 
                     encounterType.name = 'Annual Visit - Endline') AND 
                    ANY observations.concept.uuid = $0`, bmiObservationUUID)
        .filtered('TRUEPREDICATE sort(programEnrolment.individual.uuid ASC, encounterDateTime DESC) Distinct(programEnrolment.individual.uuid)')
        .filter(enc => enc.getObservationReadableValue('BMI (kg/m²)') <= 14.5)
        .map(enc => enc.programEnrolment.individual);
};

I added this ANY observations.concept.uuid = $0, bmiObservationUUID because the 'BMI (kg/m²)' field is non-mandatory. So tried to filter out the ones that don't have this field.

This takes about 9258ms, as the earlier one took about 11439ms. It seems a bit better than the earlier.

BEULAHEVANJALIN commented 3 months ago

Report Card	Before Fix	After Fix
Adolescents having moderate anaemia	6331	6212
Adolescents with severe anaemia	6223	6123
Adolescents having sickle cell disease	299	275
Adolescents having addiction	29412	321
Adolescents exited	12	14
Adolescents who dropped out of school	10822	254
Adolescents having severe malnutrition	11439	9275
Adolescents having chronic sickness	8468	488
Adolescents absent due to menstrual disorders	10486	226
Active adolescent	12	12

avniproject / Adolescent-Sewa-Rural