Upgrade jobs to work with Salesforce adaptor `v4.6.5`

[aleksa-krolls]

Background, context, and business value

Once we upgraded the SF adaptor version from v2.7.3 to v4.6.5 so that we can use Oauth, we're now seeing job run errors like this: {"message":"TypeError: Cannot read properties of undefined (reading 'includes')","name":"RuntimeError"}

Examples:

• Upsert Donors run failure
• Direct Debits run failure

From Mtuchi: there might be an issue with how the Query returns when no records returned.

The specific request, in as few words as possible

Please update the following 5 jobs to work with the latest SF adaptor version.

state.json

NOTE: We use SF Oauth on app.openfn.org (see LP API user - WFW Salesforce Sandbox Permissions (Q2 2024))

For local testing, you can use the Admin user for basic auth:

{
  "configuration": ["SEE LAST PASS: 'Admin user - WFW Salesforce Permissions Sandbox (Q2 2024)'"],
}

expression.js

1. upsertDonorsV2.js
2. upsertDirectDebits.js
3. upsertTransactionDDV2.js
4. upsertCardMasterV2.js
5. upsertTransactionCardsV2.js

To test

You can run these 3 workflows on app to test for Donors, Debit, and Credit https://app.openfn.org/projects/d84ec7c9-7b5a-4f01-a0d2-88f359deb0e2/w

manualCursor to use when testing - try "20240501" and "20240425"

[mtuchi]

I have made improvements to job codes to be able to support latest version of salesforce. For donors workflow #154 , For Credit & Debit see #153

[mtuchi]

@aleksa-krolls i am getting duplicates error for donors workflow, See run log. And for debit workflow i am getting Foreign key external ID: not found

[aleksa-krolls]

@mtuchi to unblock you here... First, I fixed the donors dupe error, but now the job is failing with a different error - see run failure. I think this error is related to when something is wrong with the external Id... so I worry we're not mapping wfw_Legacy_Supporter_ID__c properly. Can you pls take a look and ping me on Slack to discuss/if different errors pop up?

Upserting Contact with externalId wfw_Legacy_Supporter_ID__c 
Failed step Upsert Donors after 13.39s
AdaptorError: HTTP Method 'PATCH' not allowed. Allowed are GET,HEAD,POST
{"message":"HTTP Method 'PATCH' not allowed. Allowed are GET,HEAD,POST","name":"AdaptorError"}

Second, regarding the failed debit workflow run you shared - it looks like you're trying to sync debit data from the csv file 20240501, is that right? If yes, please make sure all donors are synced from the wfwi donors 20240501 file before running this debit workflow. Pls try that and then ping me on Slack if you're still running into errors.

[mtuchi]

@aleksa-krolls the mapping for wfw_Legacy_Supporter_ID__c is wfw_Legacy_Supporter_ID__c: x.PersonRef && x.PersonRef !== '' ? x.PersonRef : undefined,. I have looked at the record that is causing the error and the value for "PersonRef": "", which will make the value for wfw_Legacy_Supporter_ID__c: undefined. Can you advice how we should update the mapping ?

[aleksa-krolls]

@mtuchi I don't think it's a mapping issue... rather I think the logic has maybe been mistakenly edited. See this diagram for the donors sync workflow.

In query 2, if no Contact can be found with a SF wfw_Legacy_Supporter_ID__c that matches the CG PersonRef, then the job should move on to query 3....

However, per the run log, it looks like you're trying to upsert the Contact even though no externalId is provided... but I think rather the job should move on to query 3.

[mtuchi]

@aleksa-krolls i found the problem that was causing this behavior, It a small regression that i introduced in our salesforce adaptor. I have a fix for it here https://github.com/OpenFn/adaptors/pull/590, I will let you know once the new release is available so that you can re-test the workflow again

[aleksa-krolls]

@mtuchi when will you be ready to work on this? And do you need to pair with someone to finish this? We need to finish this before you go offline this week. Also please update the estimate or create a new issue for any work that needs to be done for this sprint... adding to top of Backlog for now and hoping you can work on it soon.

[daissatou2]

Hey @aleksa-krolls what's the latest here? Is this project ready for V2 training?

[mtuchi]

Hey @daissatou2 this PR was waiting a new salesforce adaptor v4.8.0 which was released when i was offline. My bad 🙌🏽 i should have share EOD updates before i went offline. cc @aleksa-krolls

[mtuchi]

@Aleksa Krolls i have updated salesforce adaptor version in women-for-women

Donors, Debit & Credit workflow, I have re-run the Donors workflow for wfwi Donors 20240501.csv. But i am getting a salesforce error saying

duplicate value found: wfw_Legacy_Supporter_ID__c duplicates value on record with id: 003O300000Eh8DrIAJ, Committed_Giving_ID__c duplicates value on record with id: 003O300000Eh8DrIAJ

cc @daissatou2

[mtuchi]

@daissatou2 what is the status here ?

[daissatou2]

@mtuchi in Aleksa's court