search

OpenFn / unicef-cambodia

UNICEF Cambodia - Primero Interoperability
https://openfn.github.io/unicef-cambodia/
1 stars 2 forks source link

Update how we get referrals in [Flow1] Primero -> Oscar Sync #79

Closed daissatou2 closed 2 years ago

daissatou2 commented 2 years ago

Background, context, and business value

Update [Flow1] Primero -> Oscar Sync in how we get referral data from Primero Flow 1. The same changes were made for the Primero->Progres integration with this issue.

Currently in job Flow 1, on a scheduled basis OpenFn...

Making this change because we learned that the service date is not updated when the "REFER" button is clicked on the frontend. So there is no way to get the most recent referrals with this current implementation.

The specific request, in as few words as possible

See data flow diagram: https://lucid.app/lucidchart/dd37415a-a762-4b12-97c2-dc7a1d106ad3/edit?invitationId=inv_165c7f5b-fad7-4ebb-85df-b13c2af5abeb&page=yPqq.20mBQI7#?folder_id=home&browser=icon

We will need to replace step 2 above with new steps that will...

  1. Get referrals for the cases returned in the step 1 getCases(..) request. GET /api/v2/cases/:id/referrals where id is each case record id returned in step 1. This will return JSON like this.
  2. Find new referrals: Filter for referrals where created_at_date is greater than or equal to the datetime of the last job run (or the manual cursor).
  3. For each new referral, use the service_record_id to then determine which service(s) from the step 1 we should map & send to OSCaR.

1 Service = 1 Referral... so this whole process will help us to determine how many and which services to send to OSCaR.

state.json

Primero V2 Cambodia Demo API user

adaptor

Primero adaptor Primero API docs

expression.js

Please create new job... add to V2 directory... the others in the main directory are live on prod. https://www.openfn.org/projects/p56pbe/jobs/jywaga https://github.com/OpenFn/unicef-cambodia/blob/master/jobs/V2/f1-j1-getPrimeroCases.js

For testing

The job will basically determine... Which Primero cases have been recently updated (getCases)? Which of these cases have new referrals (getReferrals for a case)? If new referrals are found, which services to send to OSCaR (using service_record_id)?

aleksa-krolls commented 2 years ago

@lakhassane please create this as a separate job.. the current version of f1-j1 job needs to work with the last_updated_at change discussed in issue #78.

Once that is resolved... @daissatou2 will test to make sure it's working.

THEN... we will implement this change, but please create a new job for this.

lakhassane commented 2 years ago

@aleksa-krolls there are a bunch of questions here. Could you help whenever online?

  1. The issue says to get cases then get referrals for each one of those case. However in the job there are two function getCases (L26 and L57). Could you clarify what is the need for each and which one should of those cases should we get referrals for ? (or both?).
  2. Also the issue talks about a Step 2. However this is not done in the job (f1-j1-getPrimeroCases) so it gets pretty confusing.

Again if the whole job needs to be reworked for a total change of logic could you clarify. That might be easier so that I can follow the lucid chart workflow.

aleksa-krolls commented 2 years ago

@lakhassane To recap... We will need to send 2 GET requests to extract Primero cases. I updated the date filters for these. Use the 2 requests to build 1 list of cases where we need to... (1) check if those cases have a service_response_types: 'referral_to_oscar. If yes, (2) check for referrals only for those services.

We updated the data flow diagram. Let us know if any further questions. This will be a slight rework of this job... make sure to make changes only to the job in the V2 directory

lakhassane commented 2 years ago

@daissatou2 test done here: https://www.openfn.org/projects/primero-oscar-cambodia-io-staging/runs/0619386b-67c9-7e08-8d58-486bb439ba5f

For info I created a new job f1-j1-getPrimeroCasesV2.js and linked that to the job in staging.

After success run it triggered the oscar sync here: https://www.openfn.org/projects/primero-oscar-cambodia-io-staging/runs/0619386c-0b70-76f7-a322-46e49996795f

aleksa-krolls commented 2 years ago

@lakhassane To recap Slack discussion... we learned that we have to use different querying syntax if searching for existing cases via oscar_number (a custom field, not a standard identifier). cc @daissatou2

Please update the GET request step that checks for existing cases in Primero as follows...

GET https://demo-kh.primero.org/api/v2/cases?query=oscar_number={oscarid}&remote=true&case_id={caseid}

Then please re-run f2-j1 to see if that works: https://www.openfn.org/projects/primero-oscar-cambodia-io-staging/runs/06194d58-2813-7452-b176-4c08f2c045e3