Validate referrals from DTP & send notification if error

[aleksa-krolls]

Background, context, and business value

This is related to the call you joined where we discussed catching errors and sending updates back to DTP. See here for the original data flow diagram, where now we want to let DTP know if the upload to Primero succeeds in step 5 in the diagram.

The specific request, in as few words as possible

I drafted this job Flow1-3 so that if the upload to Primero fails (Flow1-1 run), we will send a notification to DTP... however, this notification should ONLY be sent if DTP does not provide certain information... we do not need to send for all run failures.

If the Flow1-1 run fails because of a Primero-side issue (e.g., 400 or 500 error - see example 504 error)... then we do NOT want to notify DTP, because there is nothing they can do. The responsibility would fall on Primero team and OpenFn to resolve this error.

Here are 2 scenarios where we DO want to send notifications to DTP and dynamically update closure_reason:

1. In job Flow1-1, before we upload to Primero, we should verify that the following fields have been provided in the DTP payload sent to the OpenFn Inbox. https://www.openfn.org/projects/p5am6e/jobs/jywr9p

   
   interventiontype.progres_description,
   individuals.progres_id,
   individuals.progres_registrationgroupid,
   individuals.progres_givenname
   individuals.progres_familyname

If these fields have NOT been provided, we should throw an error and send the following message to DTP: 

"closure_reason": "Intervention referral is missing fields required for sending to Primero. Please include missing fields and re-send the request"

2. Then (if possible), if job `Flow1-1` fails because of a `TypeError` ([see example](https://www.openfn.org/projects/p5am6e/runs/rrk3z7gd)), can we also send a notification? If `TypeError` is thrown, then we should assume that there are missing fields or invalid data provided to DTP and we should send the following message to DTP. 

"closure_reason": "Intervention referral is has provided invalid data. Please review the data values shared and re-send the request."

Remember, if job `Flow1-1` fails for other reasons - OpenFn timeout, or because of a Primero error (e.g., `400` or `500` error - [see example 504 error](https://www.openfn.org/projects/p5am6e/runs/rmkdaxgk)), then we should send NOT send a notification to DTP. 

## expression.js
Job that processing referrals from DTP where we should check if all info has been provided: https://github.com/OpenFn/primero-progres/blob/master/jobs/1.upsertExtracts.js

For sending error notifications, I drafted this job: https://github.com/OpenFn/primero-progres/blob/master/jobs/1.dtpReferralFails.js
Named Flow1-3 on prod: https://www.openfn.org/projects/p5am6e/jobs/jyp94p

## trigger
Triggered by this message filter job Flow 1-1: https://www.openfn.org/projects/p5am6e/jobs/jywr9p

## state.json
Sample state for Flow1-1 (which triggers this 1-3 job): 
https://www.openfn.org/projects/p5am6e/messages/m3p7dkea (missing `nterventiontype.progres_description`)

See LP for `Primero Dadaab (open_function)` or DTP certificate

## output.json
Send [this notification](https://github.com/OpenFn/primero-progres/blob/master/sampleData/errorNotificationToDTP.json) to DTP if the right information is not provided... and dynamically update `closure_reason`. 

## TO TEST
Please run this new job against this Message on Prod... this does NOT have a service type ( `interventiontype.progres_description`) so I expect it to fail. 
https://www.openfn.org/projects/p5am6e/messages/m3p7dkea

[lakhassane]

@aleksa-krolls I implemented that new flow

[aleksa-krolls]

@lakhassane Looking great. If it's not too difficult, can we please add to the logs specifically what field is missing? Current message:

Error: Intervention referral is missing fields required for sending to Primero. Please include missing fields and re-send the request

Desired:

Error: Intervention referral is missing fields required for sending to Primero: {fieldNameMissing}. Please include missing fields and re-send the request