Bug: Possible resending text messages bug

[allibail]

Describe the bug We noticed in src>workers>job-processes.js that there could be an erroneous conditional in the dispatchProcesses function on line 285. The variable toDispatch is set to either syncProcessMap or processMap based on the truthiness of JOBS_SAME_PROCESS, and then in the for loop on line 288, p is only checked against processMap. Does this mean not every function in syncProcessMap is run? We are running a multi org deployment on lambda and want to make sure that scheduled jobs are being handled correctly.

[schuyler1d]

I believe this is fixed with this PR: https://github.com/MoveOnOrg/Spoke/pull/1796 coming in 9.1

[Frydafly]

Checking in whether this can be closed, given 9.1 @schuyler1d ?

[schuyler1d]

There IS still an issue with a bunch of dispatch processes including/especially resending. I think there are unfortunately some issues on multiple layers:

• While the code described in the initial issue is tweaked/fixed, I think there still might be some process issues in this code range: https://github.com/MoveOnOrg/Spoke/blob/main/src/workers/job-processes.js#L385-L391 and basically the logic by which we determine what things are run. The JOB_PROCESS_MAP var just above that function was a 'hack' that WFP and AFLCIO deployed to avoid running things that were broken.
• messageSenderCreator method has a lot of sensitivity to delay settings and maxCount -- we should have saner defaults that correspond better with Heroku, AWS and other environments (maybe worth consulting @mayefsky here)
• jobs.js#sendMessages has several issues as-written which can create race conditions. and locking which we need to resolve. Basically that top query can be a lot of joins and severely slow spoke sending operations and we need to review when/how that's done there or if there's a better way (maybe on-send error, we would add a cache queue that can be popped from, e.g.)