hibiken / asynq

Simple, reliable, and efficient distributed task queue in Go
MIT License
10.03k stars 716 forks source link

Support for High-Frequency Millisecond-Level Scheduled Tasks #961

Closed missish closed 1 week ago

missish commented 1 week ago

Hi asynq team,

Thank you for the fantastic work on asynq. I’m a big fan of this library, and it has been very helpful in my projects. However, there’s one feature that I believe would significantly enhance its utility: support for high-frequency, millisecond-level scheduled tasks.

Feature Request:

I’d like to request support for scheduling tasks with sub-second intervals, specifically on the millisecond level. Currently, it appears that asynq doesn’t support intervals shorter than one second. Adding this capability would make asynq a great fit for applications that require high-frequency processing.

For reference, a library like gocron has implemented this with a feature called DurationJob, which allows scheduling tasks with any duration, including millisecond-level intervals (see documentation). A similar implementation in asynq would be a game-changer for use cases requiring more precise, frequent scheduling.

Example Use Cases:

Real-time data processing pipelines High-frequency health checks or pings Rapid polling mechanisms for APIs or data sources Proposed Solution:

One way to implement this might be to enhance the Scheduler to accept durations specified in milliseconds, or add a new scheduling method that can handle sub-second intervals.

Thank you for considering this feature request! Please let me know if I can provide any additional details or if there’s any way I can contribute.

kamikazechaser commented 1 week ago

to enhance the Scheduler to accept durations specified in milliseconds

This library uses robfig/cron/v3 which doesn't support sub-second cron spec. It will be impossible to add without refactoring the entire scheduler.