upsert only if no Id is present from a previous batch

[w-andre]

This PR fixes an issue on upserting records using an external id if these records are part of a subsequent batch. In the subsequent batch the record id is added to the fields and the Salesforce API does not support upserting records while specifying a record id. Therefore the logic is changed by this PR to use the default logic to insert/update records depending on the existence of the Id field. If we have an id specified we always do an update but if we don't have an id we insert or upsert depending on an existing external id.

[FabienTaillon]

@w-andre thanks for the PR. I'm not sure I understood in which case an Id wasn't there and then is populated, would you have a simple data plan file illustrating that please ?

[w-andre]

Hi @FabienTaillon,

we are importing CPQ Price Rules and we want to import the following rule:

For this rule we need to create the rule with the default value of Conditions Met = All and then import the Price Conditions. Afterwards we can update the field Conditions Met to Custom.

To achieve this during an import we are using the following data plan:

{
  "excludedFields": [
    "CreatedDate",
    "CreatedById",
    "LastModifiedDate",
    "LastModifiedById"
  ],
  "sObjects": [
    {
      "name": "SBQQ__PriceRule__c",
      "label": "ConditionsMet_NotEquals_Custom",
      "filters": "SyncId__c != null AND SBQQ__Active__c = true AND SBQQ__ConditionsMet__c != 'Custom'",
      "orderBy": "SyncId__c",
      "externalId": "SyncId__c"
    },
    {
      "name": "SBQQ__PriceRule__c",
      "label": "ConditionsMet_Equals_Custom_Excluding_Field_ConditionsMet",
      "filters": "SyncId__c != null AND SBQQ__Active__c = true AND SBQQ__ConditionsMet__c = 'Custom'",
      "orderBy": "SyncId__c",
      "externalId": "SyncId__c",
      "excludedFields": ["SBQQ__ConditionsMet__c"]
    },
    {
      "name": "SBQQ__PriceCondition__c",
      "filters": "SyncId__c != null AND SBQQ__Rule__r.SBQQ__Active__c = true",
      "orderBy": "SyncId__c",
      "externalId": "SyncId__c"
    },
    {
      "name": "SBQQ__PriceRule__c",
      "label": "ConditionsMet_Equals_Custom_Including_Field_ConditionsMet",
      "filters": "SyncId__c != null AND SBQQ__Active__c = true AND SBQQ__ConditionsMet__c = 'Custom'",
      "orderBy": "SyncId__c",
      "externalId": "SyncId__c"
    }
  ]
}

With this plan we first import all Price Rules that do not use a Custom value for Conditions Met, then import the ones with a custom condition but without including the Conditions Met field. In this case the default value is used and this plugin tracks the returned Id for further use. Now we can import the related Price Conditions. Afterwards we are importing the Price Rule with a custom condition again but including the Conditions Met field. Since the plugin already has an Id for these records they are not inserted again but updated.

This PR updates the logic for an UPSERT using the external id, so that we are actually doing an UPDATE on the last entry in the data plan because we already have the record Id from the previous import. Without this PR the plugin would always perform an UPSERT and during the last import the Id field is also included - which will result in an error.

Regards, André

[w-andre]

Hi @FabienTaillon, I just wanted to check if you need any additional information regarding this PR?

André

[FabienTaillon]

@w-andre isn't that easier to remove the external Id from the last call ? As you said, the record has been created so the plugin is able to match the record correctly in the target org.

So basically just removing the last line:

{
  "excludedFields": [
    "CreatedDate",
    "CreatedById",
    "LastModifiedDate",
    "LastModifiedById"
  ],
  "sObjects": [
    {
      "name": "SBQQ__PriceRule__c",
      "label": "ConditionsMet_NotEquals_Custom",
      "filters": "SyncId__c != null AND SBQQ__Active__c = true AND SBQQ__ConditionsMet__c != 'Custom'",
      "orderBy": "SyncId__c",
      "externalId": "SyncId__c"
    },
    {
      "name": "SBQQ__PriceRule__c",
      "label": "ConditionsMet_Equals_Custom_Excluding_Field_ConditionsMet",
      "filters": "SyncId__c != null AND SBQQ__Active__c = true AND SBQQ__ConditionsMet__c = 'Custom'",
      "orderBy": "SyncId__c",
      "externalId": "SyncId__c",
      "excludedFields": ["SBQQ__ConditionsMet__c"]
    },
    {
      "name": "SBQQ__PriceCondition__c",
      "filters": "SyncId__c != null AND SBQQ__Rule__r.SBQQ__Active__c = true",
      "orderBy": "SyncId__c",
      "externalId": "SyncId__c"
    },
    {
      "name": "SBQQ__PriceRule__c",
      "label": "ConditionsMet_Equals_Custom_Including_Field_ConditionsMet",
      "filters": "SyncId__c != null AND SBQQ__Active__c = true AND SBQQ__ConditionsMet__c = 'Custom'",
      "orderBy": "SyncId__c"
    }
  ]
}

[w-andre]

@FabienTaillon, you're right, this would be possible. You just need to make sure that you do not have any new records in the second batch. I'm closing this PR.