search

OpenFn / grassroot-soccer

Grassroot Soccer CommCare-Salesforce integrations
https://openfn.github.io/grassroot-soccer/
0 stars 0 forks source link

Project concurrency not respected when bulk re-processing messages? #26

Closed daissatou2 closed 3 years ago

daissatou2 commented 3 years ago

Describe the bug

I was bulk reprocessing messages with the last failed runs in the GRS project and noticed that these two runs started at the same time: https://www.openfn.org/projects/pd3yk4/runs/rd76zgrp https://www.openfn.org/projects/pd3yk4/runs/r446zrnx

This resulted in duplicate records being created in Salesforce, even though we are using the Upsert function. They should have run one after the other. The first run should have created the Attendance__c record, and the second one should have found and updated the existing Attendance__c record. Instead, it looks like they ran at the same time so the second run didn't find the SF record from the first run because it had not finished creating it yet?

To Reproduce

  1. Here is a link to a failed run on OpenFn.org wihch is indicative of the bug: N/A

expression.js

Link to the job itself in Github: https://github.com/OpenFn/grassroot-soccer/blob/master/jobs/upsertCoachSessionRegister.js

state.json

https://www.openfn.org/projects/pd3yk4/messages/mrb9v6n4

aleksa-krolls commented 3 years ago

@taylordowns2000 low priority, but a funny thing we noticed... perhaps something to discuss with @daissatou2 while it's fresh in her mind sometime this week.

taylordowns2000 commented 3 years ago

Hey @daissatou2 , this is pretty strange. Seems like (at least) two separate things going on:

  1. Why did these runs both start at the same time (roughly) even though the concurrency is set to 1.
  2. How could Salesforce create two different records with the same unique, external ID.

On #2, I'm realising that we've made this CommCare_Case_ID__c field non-unique. Is that intentional? Is there any reason that we'd want to load two different attendance records into Salesforce with the exact same CommCare_Case_ID__c? While I investigate #1, a quick change here might help out.

Also, FWIW, I couldn't find the second record in Salesforce—says it was deleted. Is that expected?

image

daissatou2 commented 3 years ago

@taylordowns2000 on 2, the CommCare_Case_ID__c should be unique. And yes, we deleted the duplicate record in SF.

Can I close this one since it's being addressed here? https://github.com/OpenFn/platform-app/issues/1384

taylordowns2000 commented 3 years ago

Yes, go for it! (To confirm: did you implement that uniqueness constraint in Salesforce already?)

On Thu, Aug 5, 2021 at 12:15 PM daissatou2 @.***> wrote:

@taylordowns2000 https://github.com/taylordowns2000 on 2, the CommCare_Case_ID__c should be unique. And yes, we deleted the duplicate record in SF.

Can I close this one since it's being addressed here? OpenFn/platform-app#1384 https://github.com/OpenFn/platform-app/issues/1384

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/OpenFn/grassroot-soccer/issues/26#issuecomment-893375104, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCUBLN4N7BOROFR3DZJDDDT3JXF3ANCNFSM5BSI2SMA .