search

gocelery / gocelery

Celery Distributed Task Queue in Go
MIT License
2.37k stars 241 forks source link

Periodic tasks and future tasks #89

Open maxkleb opened 5 years ago

maxkleb commented 5 years ago

Hi, thank you for an amazing product. I've noticed that gocelery doesn't support periodic tasks and future tasks (or at least can't find it in the docs). Is this something that you plan to implement? If not can you please advice how can I implement this functionality by my own?

Thanks a lot.

yoonsio commented 5 years ago

Implementing periodic task from client side should be simple enough but I haven't looked into how celery is doing it. Are you trying to implement go client to call python celery workers?

yoonsio commented 5 years ago

@maxkleb it looks like celery.beat, a package responsible for periodic tasks, kicks off tasks at regular intervals. GoCelery workers should be compatible with these tasks. I will try to include an example. Do you also require GoCelery clients to send periodic tasks to python celery workers?

maxkleb commented 5 years ago

Hi, I'm running Go clients as well as workers. The idea is to run client service duplicated to be more fault-tolerant.

aaronn commented 5 years ago

An example would be really helpful for this. đź‘ŤI use django-celery-beat.

aaronn commented 4 years ago

Hey @sickyoon, just checking up on this since you mentioned it should be compatible . Right now my current celery instance is kicked off with celery -A myproj.celery worker --beat -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler --autoscale=10,1

However I don’t see any way I can see the —beat flag or the scheduler flag with go-celery. Am I doing this the wrong way? I haven’t configured many of these systems before.

Edit: Ah I think I see— I would have my celery beat instance still running as a python process triggering the go worker. My confusion comes from how I’ve embedded the beat service in the worker.