Flow1-2: Only send referral decisions if service date > `state.lastRunDateTime`

aleksa-krolls commented 2 years ago

Issue

Currently we send the same decision multiple times to Progres because our lastRunDateTime cursor does not filter on the service-level.

For example, we have sent this decision below 2 times (see run 1 and run 2).

Decision to send to DTP:  {
  "case_id": "47b52950-ffda-4bd9-9b92-ecba9c4dbb12#ff7ec26fe202",
  "primero_user": "primero_cp",
  "progres_interventionnumber": "612-21-PRTITV-0000042",
  "status": "rejected",
  "closure_reason": "Care arrangement already organized; not needed",
  "request_type": "ReceiveDecisionOutgoingReferral"
}

Request

Please add a filter so that if service.service_response_day_time < state.lastRunDateTime ... do NOT send this service to DTP. (This means that service might have already been sent... we only want to send services that have been updated since the last run and are therefore contain "new" decisions we want to send back to DTP.)

E.g., If state.lastRunDateTime: "2021-09-24T11:00:00.000Z", && "service_response_day_time": "2021-09-24T10:25:38.000Z" ... do NOT send this service decision onto Primero.

expression

Add onto this existing set of service filters: https://github.com/OpenFn/primero-progres/blob/586a9186a40273b4099315ba05e217badcc78fee/jobs/1.b.sendReferrals.js#L23-L34 Job in platform: https://www.openfn.org/projects/primero-progres-v4-staging/jobs/jv89rx

state

See DTP-sendDecision (open_function)credential in OpenFn: https://www.openfn.org/projects/primero-progres-v4-staging/jobs/jv89rx

Feel free to clear state & then re-run this job on platform to test... https://www.openfn.org/projects/primero-progres-v4-staging/jobs/jv89rx

aleksa-krolls commented 2 years ago

@daissatou2 fyi, this came up while testing with UNHCR yesterday. I saw that we were sending the same decision to Primero multiple times. I would appreciate your help testing tomorrow when @lakhassane has finished.

To test:

Log in as primero_cp user
Approve the pending referral for this case: https://demo-cpims-gambella.primero.org/v2/cases/0954766d-c5b1-4cd0-ae0f-fbf18be30609
Run job Flow1-2 to send your decision back to Progres
Now approve a second pending referral - see this case: https://demo-cpims-gambella.primero.org/v2/cases/b73bc2f1-c117-4d03-98f5-af5f1a2309c4
Again run job Flow1-2 to send your decision back to Progres

THEN... check the run log for step 3. It should contain only the decision from step 2. And then check the run log for step 5. It should contain only the decision from step 4. (If this run log has both decisions, then this filtering by service date is not working and we need to share this example with Mamadou.)

lakhassane commented 2 years ago

@aleksa-krolls @daissatou2 I made the change to filter with service_response_day_time. Find the 2 runs here and here

daissatou2 commented 2 years ago

@lakhassane tested this and it looks look like we are not properly getting referral updates. I set this referral to accepted at the time in the image. And ran job flow 2-1 which did not pick up any updates.

https://www.openfn.org/projects/primero-progres-v4-staging/runs/0619e428-6081-7a3d-8f09-32ca62943880

OpenFn / primero-progres