Closed daissatou2 closed 2 years ago
@grayey @taylordowns2000 Here is the long-awaited spec for rewriting the UNICEF job. Let @daissatou2 know if you want to jump on a call to walk through this today.
I estimated wrongly. This appears to be more than a 2 . Should be a 3-4. I'll update as a 4 to err on caution. (Using 5 as 4 is absent as a poker point). The next operation on the job is to begin upserting since the maps are done.
@taylordowns2000 cc: @aleksa-krolls @daissatou2
@daissatou2 any idea about this error when trying to upsert case? Complaining about certain fields.
I'm going away now. Still facing errors with upsertCase so I'm unable to test getCase. If this step is done, the next operation to updateReferrals is last.
@grayey estimates that once we get a response on the blocking question there is about 1 hour of work remaining. He was unable to work on the updateReferrals
section (a step when we send decisions) because he’s missing some information from Aicha.
@aleksa-krolls , it looks like the only commits that were missed in Emeka's branch were the two that came after b1726ca
.
Both were made on the 22nd by Aleksa, and it looks like they'll be pretty easy to reimplement.
If you give me a thumbs up, I'll get the current version from Emeka's branch onto master
and you can lay down these two commits again. Does that work?
@taylordowns2000 thumbs up! I'll then re-add my commits today
@taylordowns2000 Some updates...
Process 1
(see diagram). Given that I copied over some data transformation functions from the old job, can you please review my commit to ensure everything has been implemented optimally? https://github.com/OpenFn/unicef-cambodia/commit/da49f939ecfb39a51f0fcae2e8bd59cf2612a19e
Then, we are far from being able to test. The job doesn't look close to being done.
Process 1
- This has not been implemented correctly. Two critical things I notice so far, but I think this needs review in entirety...
L338
. Process 2
- It looks like this has not been implemented yet at all. Keep us posted on how we can support this week.
Operation 1 and 2 done, separated cases from decisions. Will pick up tomorrow AM.
@aleksa-krolls , @daissatou2 - ready for you review. please see TODOs and last commit updating comments: https://github.com/OpenFn/unicef-cambodia/commit/fd699fec81cab711e1ee24d1cc34150d542c17bf
@daissatou2 , all done! please update the job on prod to use v2.7.0
to access these new features
@daissatou2 job version updated on platform: https://www.openfn.org/projects/primero-oscar-cambodia-io-staging/jobs/jv79xn On Monday, please test
@taylordowns2000 the decision on referrals looks good.
However, in the services section, we are creating a new service instead of updating the existing one with the status. We should be updating the existing service (marked with refer again) with the Accepted
status.
Run link to sync decision back to Primero: https://openfn.org/projects/primero-oscar-cambodia-io-staging/runs/0625054e-2ad0-7bee-9e0a-59bd53349246
"external_id": "19673a21-31e6-4053-ac7d-04dee24bda6f",
@daissatou2 , this is ready to test after https://github.com/OpenFn/unicef-cambodia/commit/e64c24f8eada08d52b94be0123ce14d7671a7113
testing here is pending #107
See failed run: https://openfn.org/projects/primero-oscar-cambodia-io-staging/runs/06254481-b96e-7452-a663-c9528daaf58e
Testing here is still pending #107, see latest comments there.
Seeing this error: https://openfn.org/projects/primero-oscar-cambodia-io-staging/runs/062557f0-2f79-7a52-8cb0-10fa2cf0af1e
The run passes but there is a (node:189988) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'unique_id' of undefined
error.
You can test locally with "case_id": "54b63bb9-1d70-41b6-9f52-3e7eadd1adbc",
after running the f2-j1 job.
@daissatou2 , this error happens when no matchingService is found. I've updated the logic, please confirm that this is desired: we don't update decisions if no matching service is found: https://github.com/OpenFn/unicef-cambodia/commit/7d3b148ead7c7618eac68ebc0d7e6d2c4a77009f
@taylordowns2000 let's discuss. The job should have found a matching service for "case_id": "54b63bb9-1d70-41b6-9f52-3e7eadd1adbc
@taylordowns2000 @aleksa-krolls I've tested sending one decision from Oscar to Primero and that passed. I set up 4 cases in Primero (that need to be synced to Oscar) so I can test syncing multiple decisions, and cases from Oscar to Primero in one run.
I'll need to troubleshoot an issue where 3 of the 4 cases I created in Primero are not being synced to Oscar. See comment under #107
Feature Request
Background, context, and business value
We are rewriting the f2-j2-upsertCasesToPrimero job which upserts case and referral data fetched from Oscar to Primero.
The specific request, in as few words as possible
See the data flow diagram. LucidChart login is in LP.
The f2-j2-upsertCasesToPrimero should have two main functions: 1)send Oscar cases (with or without services) to Primero 2) send Oscar decisions on Primero referrals to Primero. Follow the steps below for each function of the job.
Process 1: Sending Oscar cases to Primero
Given an array of Oscar cases that match the criteria below, upsert cases in Primero using
oscar_number
orcase_id
as the UUID.If
oscar.resource != 'Primero' || oscar.resource===undefined ||(oscar.resource == "Primero" && oscar.status=== "Referred")
, then this is an Oscar case we need to sync.Log cases that meet this criteria as
"Cases without decisions ".
Then
upsertCase()
using the mapping specifications. In column A we have marked the fields that should be sent in this step and you can filter by this column. -Ifoscar.external_id
isundefined
, upsert the Primero case and related services using "oscar_number": "oscar.global_id" as the external_id upsert key. -Ifoscar.external_id
is defined, upsert Primero case and related services using "case_id": "oscar.external_id" as the external_id upsert key.*We assume 1 oscarService = 1 primeroService. For each record in `oscar.services[]
, we will upsert 1 Primero
case.services_section`. SEE MAPPING SPECS FOR SERVICE MAPPING RULES.**Process 2: Sending Oscar decisions to Primero
Given an array of Oscar cases that match the criteria below, upsert cases AND referrals in Primero using
case_id
as the UUID.oscar.resource = "primero" && oscar.status!= "Referred"
this is a decision on a Primero referral."Oscar decisions"
.Referral-level update For each matching case ...
GET /cases/:id/referrals using the oscar.external_id
. See sample JSON response.For each Oscar service where
oscar.services.enrollment_date = null
loop through these referrals until we find the matching referrals whereservice_record_id
="oscar.services.uuid"
updateReferral(...)
to PATCH an update to the Primero status field with what is in the referral_status field from the Oscar response. PATCH /cases/:record_id/referrals/:id See sample JSON payload for request bodyMappings notes for referral level update: if we get value "Accepted" from Oscar
status
field, send to Primerostatus: 'accepted'
if we get value "Exited" from Oscarstatus
field, send to Primerostatus: 'rejected'
Case-level update:
upsertCase()
using the mapping specifications.case_id: oscar.external_id
services_section[*].unique_id: o.services.uuid
services_section[*]. referral_status_ed6f91f = accepted_850187 || rejected__74769
based onoscar.status
state.json
Primero: Oscar (to generate state):
{ "configuration": "Oscar Staging (api only user)", "data": "Data will be generated from this job" }
{ "configuration": "Primero V2 Cambodia Demo (openfn_cp_user)", "data": "Run F2-J1 to generate state" }
adaptor
language-primero
expression.js
To re-write this job... consider creating a new file to preserve the original expression. [f2-j2-upsertCasesToPrimero](https://github.com/OpenFn/unicef-cambodia/blob/master/jobs/f2-j2-upsertCasesToPrimero.js)