search

OpenFn / lightning

OpenFn/Lightning ⚡️ is the newest version of the OpenFn DPG and provides a web UI to visually manage complex workflow automation projects.
https://openfn.github.io/lightning/
GNU Lesser General Public License v3.0
126 stars 34 forks source link

Rate limiting on webhook requests #2219

Open josephjclark opened 3 months ago

josephjclark commented 3 months ago

We've hit on a problem where Lightning is returning too many requests when triggering new workflows.

We have a use-case where we're sending an array of records into a job, and for each item, triggering a new workflow run. The implementation team specifically does not want this to be a batch operation.

I think the error is occurring for less than 50 records being sent within a few ms. This doesn't feel like especially high volume

This workflow is triggered by a cron job at midnight, a busy time for the server. So maybe this is adding extra load at a busy time and that's why the rate limiter is kicking in.

josephjclark commented 3 months ago

@hunterachieng when you have a chance could you link to a run that shows the too many requests error?

christad92 commented 2 months ago

@hunterachieng any luck with this?