search

OpenFn / lwala

2 stars 3 forks source link

New job for creating `Service__c` records in Salesforce #56

Closed aleksa-krolls closed 2 years ago

aleksa-krolls commented 2 years ago

Background

Lwala CHWs collect data in CommCare and want to sync these with Salesforce for reporting across CHW areas. When CHWs visit Households, they will register a new "case" if the person needs a referral or follow-up. These cases should be registered in Salesforce as Service__c records, which link to the related Person__c (parent table in SF) visited.

Request

Upsert 1 Service__c record in Salesforce for each Message (aka case update) received from CommCare.

See OpenFn Map tab for mapping specifications.

adaptor

salesforce

expression

New Upsert_Service.js job

state

configuration: See LP for Lwala Salesforce Sandbox MOTG (Dec 21) data: https://github.com/OpenFn/lwala/blob/master/sample_data/taskUpdateCase_Malaria.json

Testing Notes

Feel free to create data in the Sandbox as you wish. Successful run of this message should create 1 new Service.

aleksa-krolls commented 2 years ago

@elias-ba can you please prioritize this issue today? I will be online midday to check-in on any questions you might have. Thank you!

elias-ba commented 2 years ago

@aleksa-krolls I am a little bit confused by the structure of this data file https://github.com/OpenFn/lwala/blob/master/sample_data/taskUpdateCase_Malaria.json. Shouldn't it contain arrays of cases ? What's exactly a case in here ? The properties object ? a mix of the properties and some other values like the case_id ? I am using the mapping sheet but it's huge and I wanna make sure I am not misunderstanding the content of the data file. Could you help clarify this for me whenever you have the chance to be online ?

elias-ba commented 2 years ago

I also see things like this hardcode default value - see comments in the mapping and I don't quite understand what they mean. Does it mean that I should consider that this value doesn't exist in the data and that I should hard code it ? or does it mean that the field name is RecordTypeID but may not be present every time in the data.

elias-ba commented 2 years ago

Hey @aleksa-krolls we're done implement this. Here is a link to the expression file: https://github.com/OpenFn/lwala/blob/master/commcare-salesforce-jobs/Upsert_Service.js. We didn't create the job in platform cause we do not have information on what the trigger is. Please test it and let us know if there's any feed-back. cc: @taylordowns2000

aleksa-krolls commented 2 years ago

@elias-ba This job has been working well, but one change request has surfaced.

Background & sample state

It turns out CommCare will send us 2 kinds of state.data.indices.parent.case_type Messages:

  1. "state.data.indices.parent.case_type": "Person": https://www.openfn.org/projects/lwala-chw-support/messages/061c5d9e-dae5-7f2a-af03-a953a8072992
  2. "state.data.indices.parent.case_type": "Case": https://www.openfn.org/projects/lwala-chw-support/messages/061d4b13-549a-7310-9721-966750fa8961

Request

If (1) "Person" case_type, then...

If (2) "Case" case_type, then...

https://github.com/OpenFn/lwala/blob/b540ec64225e5119532d1c395d483a825f24953b/commcare-salesforce-jobs/Upsert_Service.js#L32-L43

Note

Other than these 2 parent.case_type differences, all of the other mappings are the same... so I'm not sure if it makes to have a different upsertIf(...) operation for each type, or if there is a way we can make only these 2 mappings dynamic. cc @taylordowns2000

Example job mappings

  1. For this sample "state.data.indices.parent.case_type": "Person": https://www.openfn.org/projects/lwala-chw-support/messages/061c5d9e-dae5-7f2a-af03-a953a8072992 ... the mappings should look as follows: 

    relationship(
  "Person__r",
  "CommCare_ID__c",
  dataValue("indices.parent.case_id")
),
//NO mapping for Parent_Service__r

  2. For this sample "state.data.indices.parent.case_type": "Case": https://www.openfn.org/projects/lwala-chw-support/messages/061d4b13-549a-7310-9721-966750fa8961 ... the mappings should look as follows: 

    relationship(
"Parent_Service__r",    //here we map this field
 "Service_UID__c",
dataValue("indices.parent.case_id")
),
relationship(
  "Person__r",
  "CommCare_ID__c",
  {response CommCare_ID__c from SF select query - see point 2 above}
)
elias-ba commented 2 years ago

@aleksa-krolls do you have any idea about this error ? INVALID_FIELD: Foreign key external ID: a030800001zr4hq not found for field CommCare_ID__c in entity Person__c. It seems like the value a030Q000008XyXV doesn't exist.

aleksa-krolls commented 2 years ago

@elias-ba when sharing an error, please confirm which state you are using to test because I'm not sure how you got that error mentioned in your comment... to avoid such an error I had already hardcoded an ID in Line 26 for sandbox testing.

As I commented on Slack- I added the missing person to the sandbox, can you please proceed with the changes? Fyi, you will continue to hit the error in this until you implemented the change I requested: https://www.openfn.org/projects/lwala-chw-support/runs/061d8512-f78e-7c81-b159-e603d6f07577

aleksa-krolls commented 2 years ago

@elias-ba see above for my comment from last week, and lmk if you have more questions or if a quick call would help ... would be great to start testing this morning.

aleksa-krolls commented 2 years ago

@taylordowns2000 I've just ran a test, and it's failing when form.indices.parent.case_type === "Case". https://www.openfn.org/projects/lwala-chw-support/runs/061dc0ee-478a-7b00-9e77-524777293d91

See L14 in run log where the CommCare_ID__c is undefined even though a record was found in the SOQL query. Can you take a look?