NangoHQ / nango

A single API for all your integrations.
https://www.nango.dev
Other
4.61k stars 414 forks source link

fix: missing created task events #2364

Closed TBonnin closed 3 months ago

TBonnin commented 3 months ago

When tasks are being scheduled by the scheduling worker a message is posted to the scheduler that searches for the tasks and emit a CREATED event per task. The message was posted within a transaction which led to cases where the scheduler search query would return nothing because it would be executed before the transaction was committed and the tasks actually visible. I was not able to reproduce this issue locally. I imagine the transaction commit always happens fast enough with a local db.

The lack of CREATED task events caused some delays (up to 30-40seconds) of the scheduling of tasks because this event allows the dequeue long polling to return early when a task is created instead of waiting for timeout.

This commit modifies the scheduling worker so the message is posted once the transaction is finished/committed