search

OpenFn / gife

0 stars 0 forks source link

update `salesforce` jobs to use `bulkQuery` #12

Closed mtuchi closed 11 months ago

mtuchi commented 1 year ago

Summary

A new helper function for querying records in bulk is now available of salesforce adaptor v4.2.0.

Update jobs/getDeletedCampaignMembers.js and jobs/getCampaignMembers.js to use the new bulkQuery function. and remove the offset query.

Important thing to note, the result of the bulkQuery() function will be in state.data not in state.references So you have to update const campaignMembers = state.references.map(ref => ref.records).flat(); to use const campaignMembers = state.data;

aleksa-krolls commented 1 year ago

@bryceVeraSolutions as @mtuchi flagged on Slack - please hold working on this issue until we cut a new Adaptor release on Monday

mtuchi commented 1 year ago

@bryceVeraSolutions new version of salesforce adaptor is available now, @openfn/language-salesforce@4.2.1 You can proceed now with implementation.

You can clean the existing adaptors version by running openfn repo clean first before running openfn workflow.json -i -o tmp/output.json

mtuchi commented 1 year ago

Hiya @bryceVeraSolutions , sorry i forgotten to mention a crucial details about the bulkQuery. You need to specify apiVersion: "59.0" in your salesforce credentials

mtuchi commented 1 year ago

Regarding bulkQuery() function failing for aggregate queries It doesn't seem to be the issue with the adaptor function, rather it's limitation with Bulk API 2.0 It has a limitation on Parent-to-child relationship queries. (Child-to-parent relationship queries are supported.) cc @bryceVeraSolutions

mtuchi commented 12 months ago

First workflow is done on this PR https://github.com/OpenFn/gife/pull/13 Thanks @bryceVeraSolutions