search

timgit / pg-boss

Queueing jobs in Postgres from Node.js like a boss
MIT License
2.09k stars 158 forks source link

batchSize workers don't autocomplete jobs when the promise resolves successfully #312

Closed tcoats closed 1 year ago

tcoats commented 2 years ago

batchSize workers don't autocomplete jobs when the promise resolves successfully. Is this a conscious decision, or overlooked functionality? It would align with the normal worker if the async function returns successfully to indicate that all batched jobs are complete.

https://github.com/timgit/pg-boss/blob/8d34ab7426fb8c45e82df2b08cda7756ee63e4ea/src/manager.js#L217

timgit commented 2 years ago

This was intentional so I could more closely track completion times per job. However, after you opened this, I realize the larger the batch, the less helpful it becomes to track individual completions, since all jobs in the batch share the same start time by design.

codingedgar commented 2 years ago

Thanks @tcoats, I was also a bit confused by how/when the job returned [job] and when job, and spend about 2h debugging why it did not complete (but did expire and fail) jobs when batchSize was specified.

I'm guessing for v7.0.2, if we have batchSize then we have to manually complete each job?