Closed daissatou2 closed 2 years ago
@lakhassane this one is ready to be worked on, but it's a little tricky. So let @daissatou2 and I know if you need to talk through it.
@lakhassane Your estimate was 3 hours... how is this going? What's the status? Flagging that UNICEF may need to revoke our access today for the week as they work on data migration this week, so if you are unable to authorize... we now might need to wait until next week to finish and test if you didn't get this done yesterday.
Testing is pending Primero data migration.
Step 1 from above is working properly but Step 2 is not working
Currently, instead of updating the referral_status_5fe9c1a
field on the existing service, the job is creating a new service and still NOT adding the status on that field either.
The job should be updating the referral_status_5fe9c1a
field on the existing service using the steps outlined above for STEP 2.
We did some investigating and here is what is currently happening:
The f2-j2 job is using the same service_record_id
to find the unqiue_id
for all cases and referrals recieved from Oscar. See lines L297 and L321… service unique_id
is the same: 3e3af14a-12f2-4717-9b43-442e102fc890 for two different cases.
https://openfn.org/projects/primero-oscar-cambodia-io-staging/runs/0622a2ec-0810-76fb-b558-f9d65a0cbeb1
Is this service unique_id
being written to 1 constant, rather than finding the service_record_id
/ unique_id
for EACH relevant service on EACH case recieved from Oscar: https://github.com/OpenFn/unicef-cambodia/blob/master/jobs/V2/f2-j2-upsertCasesToPrimero.js#L261
Here's what should be happening:
The job should be finding the service_record_id
/ unique_id
for EACH relevant service on EACH case recieved from Oscar using the steps outlined above and in the the data flow diagram.
Hello @grayey how is this coming along? Any updates?
@daissatou2 , this is ready for your testing. please note the TODO for you: https://github.com/OpenFn/unicef-cambodia/blob/master/jobs/V2/f2-j2-upsertCasesToPrimero.js#L251-L254
Once you're happy with how the tests are running, you can remove that TODO and restore the if/throw.
@grayey , your intuition was spot on but in the end later operations were referring to state.service_record_id
- a string. In reality, there can be multiple cases at that point in the job and each will have its own unique service_record_id. (@aleksa-krolls and @daissatou2 , thanks for helping me see the r'ship between these.)
So.... as we go along we build a little Case:Service map. After the first loop where we update referrals I've added a log telling us what we've got for later use. Each key is a case_id
and each value is a service_record_id
:
Referrals updated. Case:Service ID map = {
'cc803894-82b2-46c7-82fa-7c0ed50c9a9f': '4d770705-db07-4979-a46c-ee6cff572a65',
'ed4c6461-1a7f-40b9-b857-cf22acddbe04': '07ac46e9-3725-4ef1-9167-127059cfff2e'
}
Finally, because of the way this job has been written, we've got to "pass along" the case_id
to a bunch of different generator functions later in the job. Note my change to mapKeysToServices:l368
and classifyServices:l400
Hope this all makes sense. Holler if you want to chat through this later today. @grayey , It's still worth a refactor, IMO. Let's discuss how we can do that when you're back online at 3pm... assuming Aicha is happy with how the testing goes.
Thanks guys! The unique_id
issue seems to be resolved.
However, according to the logs in this run, we are not sending the services_section[*].referral_status_5fe9c1a
field to Primero. So the service status is actually not being updated in Primero.
See STEP 2 in the original issue above.
Mappings notes for service level update:
if we get status: Accepted
from Oscar, send services_section[*].referral_status_5fe9c1a: 'accepted_270501'
to Primero
if we get status: Exited
from Oscar, send services_section[*].referral_status_5fe9c1a: 'rejected_412652'
to Primero
@daissatou2 , ready for testing! i've added 2 TODO
blocks for @grayey ... there's some funky stuff going on with those mapKeysToServices
and classifyServices
functions.
ok @taylordowns2000. Taking a look now.
Closing this issue. As discussed, @daissatou2 and I will prepare new specs for a new job version that will allow us to drop a lot of this legacy logic.
Feature Request
Background, context, and business value
For Flow 1 Primero -> Oscar, Oscar does not send the original Primero
service.unique_id
in their decision payload. This means we have to add additional steps and get creative to find the relevant Primero service which needs to be updated with the decision.For example: Instead of something like this which includes the
service.unique_id
, we receive this.The specific request, in as few words as possible
See the data flow diagram.
Step 1: After receiving a decision from Oscar, the V2 f2-j2-upsertCasesToPrimero job should follow the steps below to find the right referral to update.
1.For each case where
resource
= "Primero", Oscar will send a list of services inservice[*]
. Whereservice[*].enrollment_date = null
, save the Oscarservices[*].name
in anoscarReferral
array.external_id
. Use theexternal_id
ascase_id
ingetCases()
.service_response_type
= referral_to_oscar and 2)referral_status_5fe9c1a
is not defined OR does not equal neither 'accepted_270501' norrejected_412652
. This will return a list ofservices
referred to oscar.oscarReferral
, find the matching Primero service as follows mappingp.service_type: o.name
.services_section[]. service_type
== the mapped version ofserviceMap[oscarReferral]
from the original oscar response. The mapping table is already in that job.service_type
, log an error.services_section[].unique_id
asservice_record_id
. (You will use this variable again below).GET /cases/:id/referrals
using thecase_id
from step 2 above. See sample JSON response.service_record_id
= theservice_record_id
saved in step 5.Mappings notes for referral level update: if we get value "Accepted" from Oscar
status
field, send value "accepted" to Primerostatus
field if we get value "Exited" from Oscarstatus
field, send value "rejected" to Primerostatus
fieldStep 2: After updating the Primero
referral
, THEN also map the Oscar decision to the Primeroservice
.Use the
service_record_id
from step 5 above to update the service with the decision.add a services_section[] mapping for Primero
referral_status_5fe9c1a
here.service_record_id
=services_section[*].unique_id
status
field from the Oscar response. See mappings below.Mappings notes for service level update: if we get
status: Accepted
from Oscar, sendservices_section[*].referral_status_5fe9c1a: 'accepted_270501'
to Primero if we getstatus: Exited
from Oscar, sendservices_section[*].referral_status_5fe9c1a: 'rejected_412652'
to Primerostate.json
Primero:
{ "configuration": "Primero V2 Cambodia Demo API user", "data": "Run F2-J1 to generate state" } Oscar (to generate state):
{ "configuration": "Oscar Staging (api only user)", "data": "Data will be generated from this job" }
adaptor
Primero adaptor Primero API Docs
expression.js
f2-j2-upsertCasesToPrimero
Toggl
UNICEF Cambodia