salesforce: update API

[josephjclark]

New API design for salesforce, including adding composeNextState and removing old code.

Delivery Plan

We should break this down into three parts:

• Remove old/unused functions, fix tests
• Use composeNextState and fix state.references, update tests
• Update API to new signatures, with basic docs changes

Each part should be treated like its own story. It should have its own PR which is reviewed and merged into an epic branch called epic/salesforce.

Overview

All functions should use composeNextState and write to state.data.

references behaviour should be preserved, but state.data should be the primary usage from now on.

We should probably go on and fix #509 here because we're going to be working so close to that problem.

Only basic documentation is included in this issue. We'll do a deep dive on docs later.

We are going to be bold and REMOVE callbacks entirely from the API :scream: This may result in worse performance for this adaptor version on v1, but I think that's acceptable. And then() support, which is coming super soon, will empower this adaptor on v2.

It makes sense to use the new expandReferences API, but that's not technically part of the issue.

Note that the HTTP helpers (request, get, post) should use connection.request. Docs should make it super clear that these helpers only call salesforce APIs. Later, we will add a generic HTTP API with common (see #668)

This work should only have minimal unit tests. We'll do a deep dive on tests in #666

Errors should be thrown, not caught and logged , unless there is any specific messaging that can be added

API

bulkQuery(query, options)

bulk(operation, sObjectName, records, options)

create(sObjectName, records) // use records instead of attrs. Default to batch.

describe(sObjectName)

describeAll()

destroy(sObjectName, ids, options) // rename attrs (which is wrong) to ids

insert(sObjectName, records)

query(soql, options)

request(url, options)

get(url, options)

post(url, data, options)

retrieve(sObjectName, id)

update(sObjectName, records)

upsert(sObjectName, externalId, records)

toUTF8(as-is)

fnIf() // exported out of common

Code Removal

To be clear, the list above is the whole API. The following functions should all be removed.

• cleanupState()
• steps()
• reference()
• createIf(), upsertIf()
• describeAll() maybe?
• remove axios
• remove beta from common
• remove relationship

[josephjclark]

This may fix #644

[josephjclark]

This is a huge issue.

We should create an epic/salesforce branch and merge steps into it.

I strongly recommend we do this over several branches. Each one should be opened as a PR, reviewed, and merged into the epic

1. Remove old/unused functions, fix tests
2. Use composeNextState and fix state.references, update tests
3. Update API to new signatures, with basic docs changes

[mtuchi]

Just a thought exercise 🧠 , Should we keep exporting the following function from @openfn/language-common 🤔 ?

• dataPath,
• dataValue,
• field,
• fields,
• referencePath
• source,
• sourceValue,

[mtuchi]

How important is export const relationship =(relationshipName, externalId, dataSource)=>({ [externalId]: dataSource }) ? 🤔 Could this be a fix for {externald.abc: '123'} vs {externalId: { abc: '123'}}

[mtuchi]

Good start today, i have done lots of cleanup but still work in progress. I will keep at it tomorrow

[mtuchi]

Okay today went by so quick, i haven't done much in this issue but i have manage to start clearing the callbacks in each function. I need another programming block tomorrow tidy this up

[mtuchi]

Hiya @aleksa-krolls , I don't think we need to create another issue as this issue is technically done, I need to create a changeset and update the PR description #682. I will need approx 1 hrs to finish the remaining tasks

Notes For Next Step:

• [ ] Create a changeset file that has a list of what has been changed, removed, added
• [ ] Update the PR notes for Joe

[josephjclark]

@mtuchi please keep the changeset really BRIEF! I would rather ask you to add detail than to remove it

commcare 2.0 is a really good model for the right level of detail