[BUG] Salesforce: New Updated Field on Record triggers broken with certain Object-Field permutations

[malexanderlim]

Describe the bug Two of our registry Salesforce triggers only appear to be partially working - specifically, both the webhook and polling version of the trigger "New Updated Field on Record" only work in some cases.

Certain object-field permutations fail.

1. New Updated Field on Record (Instant, of Selectable Type)
2. New Updated Field on Record (of Selectable Type)

To Reproduce

Use the account in shared 1P, michael@pipedreamsand... Example workflow with working and broken triggers: https://pipedream.com/new?h=tch_3Z6fVy

Working Trigger configuration:

Contact/title

New Updated Field on Record (Instant, of Selectable Type)

1. Object Type: Contact Field Type: Title In the Salesforce instance, edit an existing contact, and change the Title. You should see the event fire successfully.

Broken Trigger configurations

Contact/title (polling), contact/email (webhook and polling)

1. The same object/field permutation does not work on the polling version: New Updated Field on Record (of Selectable Type)

New Updated Field on Record (Instant, of Selectable Type)

2. Object Type: Contact Field Type: Email A nondescript error, Error: [object Object] at s._deployWebhook (/var/task/node_modules/salesforce-webhooks/dist/index.js:1:138666) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async s._createWebhookWorkflow (/var/task/node_modules/salesforce-webhooks/dist/index.js:1:138844) at async Object.activate (file:///var/task/user/sources/common-instant.mjs:34:23) at async /var/task/index.js:95:13 at async captureObservations (/var/task/node_modules/@lambda-v2/component-runtime/src/captureObservations.js:28:5) at async exports.main [as handler] (/var/task/index.js:60:20) is returned when testing the Contact/Email permutation.

   

3. This is also broken with the polling source, as no events fire.

Expected Behavior:

1. I expect the polling trigger to fire an event when the field that is being watched on an object is changed.
2. I expect the webhook (instant) trigger to return an event for valid permutations, and return a more descript error message if there is something wrong with the setup.

Notes: The Salesforce setup is complex and somewhat difficult to navigate; I added step-by-step instructions detailing the permissions required to configure webhooks in our documentation.

[malexanderlim]

This may be related to a prior issue raised here: https://github.com/PipedreamHQ/pipedream/issues/6130

[jcortes]

Hi @malexanderlim As far as I can tell there is a problem with the SalesforceWebhooks sdk client. It's throwing the error right here where it returns something like this in the SOAP response:

...
        <runTestsResult>
...
          <failures>
            <id>01pHp00000SOHoQIAX</id>
            <message>System.DmlException: Update failed. First exception on row 0 with id 003Hp00002rlGZiIAM; first error: INVALID_EMAIL_ADDRESS, Email: invalid email address: nullnull: [Email]</message>
            <methodName>testBatch</methodName>
            <name>SW_Test_d768a3440ba1b19b7132f256340a8323</name>
            <namespace xsi:nil=\"true\"/>
            <stackTrace>Class.SW_Test_d768a3440ba1b19b7132f256340a8323.testBatch: line 17, column 1</stackTrace>
            <time>1215.0</time>
            <type>Class</type>
          </failures>
          <failures>
            <id>01pHp00000SOHoQIAX</id>
            <message>System.DmlException: Update failed. First exception on row 0 with id 003Hp00002rlGZmIAM; first error: INVALID_EMAIL_ADDRESS, Email: invalid email address: nullnull: [Email]</message>
            <methodName>testSingle</methodName>
            <name>SW_Test_d768a3440ba1b19b7132f256340a8323</name>
            <namespace xsi:nil=\"true\"/>
            <stackTrace>Class.SW_Test_d768a3440ba1b19b7132f256340a8323.testSingle: line 31, column 1</stackTrace>
            <time>555.0</time>
            <type>Class</type>
          </failures>
...
        </runTestsResult>
        <success>false</success>
...

I had to build the lib in order to see that error and run it locally with this code:

import sfw from "./dist/index.js";

const { SalesforceClient } = sfw;

const instance = "pipedream7-dev-ed.develop.my";
const authToken = "CURRENT_AUTH_TOKEN";
const apiVersion = "50.0";

const client = new SalesforceClient({
  apiVersion,
  authToken,
  instance,
});

console.log("client", client);

const promise = client.createWebhook({
  endpointUrl: "https://eoy4i49jpfgqxww.m.pipedream.net",
  sObjectType: "Contact",
  event: "updated",
  secretToken: "1234567890rewasfreage",
  fieldsToCheck: [
    "Title",
  ],
  fieldsToCheckMode: undefined,
  skipValidation: true,
});

promise
  .then(console.log)
  .catch((err) => {
    if (err.response?.data) {
      console.log(JSON.stringify(err.response.data, null, 2));
    } else {
      console.log(err);
    }
  });

I guess this would be a blocker at the moment!

[jcortes]

Waiting on https://github.com/jverce/salesforce-webhooks/pull/19 to be reviewed

[vunguyenhung]

Hello everyone, I have tested this PR and there're some test cases failed or needed improvement.

Please check the test report below for more information https://vunguyenhung.notion.site/BUG-Salesforce-New-Updated-Field-on-Record-triggers-broken-with-certain-Object-Field-permutations-0eab412d5a804edd9b5ea605a27e505b

[jcortes]

Hi @vunguyenhung I've fixed the following:

1. Events are now consistent with number of modifications/creation
2. In the case of the polling source updated-field-on-record I had to change the summary because we are fetching data from the History tables not the Object itself, so that means we have only access to the columns of the History table and if we try to fetch records from the Object that would be more expensive in terms of requests and that's something the user is concern with at the moment, so I'm just displaying the Id of the Object in the summary and not the Name.
3. In this PR I'm not fixing the Webhook sources because the bug is in the Client lib but that's something @jverce is helping us to fix

CC @malexanderlim

[vunguyenhung]

Hi @jcortes, this ticket should be closed when all issues resolved, or else it might confuse others. I would suggest to move this back to In Progress or Blocked until the upstream library is resolved. What do you think?

[malexanderlim]

I broke out the other feature to reduce the number of API calls for the polling triggers here: https://github.com/PipedreamHQ/pipedream/issues/9479

I'd like to try to get this part shipped, and we can tackle the object-field permutation issue separately.

[daniel-adayev-prefect]

Hi, I recently had this issue . We got around the issue by setting up salesforce flow to send http requests to pipedream internally on record create or updates. This gives more visibility on api usage volume, increases speed of salesforce --> pipedream trigger, and saves on cost.

[malexanderlim]

Hi @daniel-adayev-prefect - which trigger were you using specifically, and at what polling cadence? I saw your initial note and want to ensure that there isn't a bug with the polling version of the trigger, as the trigger should only poll at the cadence that you define.

Thanks for the feedback as well around Salesforce Flow - happy to surface that as an option in our documentation as well.

[daniel-adayev-prefect]

Hi @daniel-adayev-prefect - which trigger were you using specifically, and at what polling cadence? I saw your initial note and want to ensure that there isn't a bug with the polling version of the trigger, as the trigger should only poll at the cadence that you define.

Thanks for the feedback as well around Salesforce Flow - happy to surface that as an option in our documentation as well.

Hi @malexanderlim, thanks for the speedy response and sorry for jumping the gun and posting about the issue before trying to replicate it. We initially assumed it was due to the salesforce polling trigger in pipedream, but after some testing trying to replicate the issue in salesforce sandbox, we found out that it was caused by a different oauth connection that was sending thousands of api calls to salesforce and using up our entire api call allowance. This in turn prevented the pipedream salesforce polling triggers from working.

At this point, the pipedream salesforce polling triggers work great now that we've removed the erroneous OAuth app. However, when setting up the "record updated instantaneous trigger", we get an "Error: [object Object] at async" when trying to save the trigger. This isn't much of an issue though as we've set up a trigger inside of salesforce to pick up field updates and push them to pipedream.