hibiken / asynq

Simple, reliable, and efficient distributed task queue in Go
MIT License
9k stars 659 forks source link

[FEATURE REQUEST] Custom ticker value for heartbeater duration #889

Open RiskyFeryansyahP opened 1 month ago

RiskyFeryansyahP commented 1 month ago

Is your feature request related to a problem? Please describe. Currently, in our production environment, I have 9,000 scheduled entries that run once every week and once every month. However, I've noticed that the CPU utilization on my Redis spikes continuously. You can see this in the image below.

When I investigated the reason for these CPU spikes, I disabled the scheduler in asynq, and the CPU usage returned to normal. While examining the code for the scheduler in asynq, I found that the heartbeater inside asynq runs every 5 seconds (hardcoded) using a ticker. This process involves scanning through cron entries and performing WriteSchedulerEntries.

Screenshot 2024-06-02 at 15 43 43

Describe the solution you'd like I have an idea to configure the ticker duration based on SchedulerOpts. This way, the ticker duration is not hardcoded and can be defined through configuration when initializing NewPeriodicTaskManager.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

RiskyFeryansyahP commented 1 month ago

Hi @hibiken, if you agree with my proposal, I would be happy to implement it πŸ˜„

RiskyFeryansyahP commented 1 month ago

Hi @kamikazechaser, if you have time, please check my proposal request πŸ™

RiskyFeryansyahP commented 4 days ago

Any update guys?