taskiq-python / taskiq

Distributed task queue with full async support
MIT License
806 stars 50 forks source link

Cron scheduled tasks cleanup on TaskiqScheduler start #297

Open sector119 opened 7 months ago

sector119 commented 7 months ago

Hello,

After scheduler restart all cron scheduled tasks are getting duplicated... Should we remove all scheduled tasks on scheduler startup like in code bellow? Right now only time (not cron) scheduled tasks are removed on "post_send", but if scheduler was killed or stopped before task was delivered to the worker it wouldn't be deleted too

Right now I cleanup scheduled tasks at TaskiqScheduler method

async def startup(self) -> None: for source in self.sources: tasks = await source.get_schedules()

    for task in tasks:
        await source.delete_schedule(task.schedule_id)
s3rius commented 5 months ago

So, you can use labeleschedulesource for static crons. It doesn't store anything anywhere. It reads all available tasks and creates schedules based on tasks labels. You can add it along redis-schedule-source to enable this functionality.

But if you want to add them dynamically and use other schedule_sources, then that's the only way.