search

hibiken / asynq

Simple, reliable, and efficient distributed task queue in Go
MIT License
10.04k stars 716 forks source link

[FEATURE REQUEST] Custom ticker value for heartbeater duration #889

Open RiskyFeryansyahP opened 5 months ago

RiskyFeryansyahP commented 5 months ago

Is your feature request related to a problem? Please describe. Currently, in our production environment, I have 9,000 scheduled entries that run once every week and once every month. However, I've noticed that the CPU utilization on my Redis spikes continuously. You can see this in the image below.

When I investigated the reason for these CPU spikes, I disabled the scheduler in asynq, and the CPU usage returned to normal. While examining the code for the scheduler in asynq, I found that the heartbeater inside asynq runs every 5 seconds (hardcoded) using a ticker. This process involves scanning through cron entries and performing WriteSchedulerEntries.

Screenshot 2024-06-02 at 15 43 43

Describe the solution you'd like I have an idea to configure the ticker duration based on SchedulerOpts. This way, the ticker duration is not hardcoded and can be defined through configuration when initializing NewPeriodicTaskManager.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

RiskyFeryansyahP commented 5 months ago

Hi @hibiken, if you agree with my proposal, I would be happy to implement it 😄

RiskyFeryansyahP commented 5 months ago

Hi @kamikazechaser, if you have time, please check my proposal request 🙏

RiskyFeryansyahP commented 4 months ago

Any update guys?

Aldiwildan77 commented 4 months ago

Is there any update for this @hibiken?

Your issue seems fit to me, I also had this when I had a thousand jobs