CELERYBEAT_SCHEDULE Tasks not picked up by the scheduler

[ShipraShalini]

I am running a simple test to print hello every 30 sec using celery 4.0.2 in my django app.

@app.task()
def test():
    print("hello")

My celery config is:

CELERY_BROKER_URL = REDIS_CELERY_BROKER_URL
CELERY_RESULT_BACKEND = REDIS_CELERY_RESULT_URL  # 'sqs;//' no support for SQS result backend for now
CELERY_MONGODB_SCHEDULER_DB = MONGO_DB
CELERY_MONGODB_SCHEDULER_COLLECTION = "schedules"
CELERY_MONGODB_SCHEDULER_URL = MONGO_URL
CELERY_RESULT_SERIALIZER = 'json'
CELERY_ENABLE_UTC = True
CELERY_DEFAULT_QUEUE = CELERY_DEV_QUEUE
CELERY_QUEUES = {
    CELERY_DEFAULT_QUEUE: {
        'exchange': CELERY_DEFAULT_QUEUE,
        'binding_key': CELERY_DEFAULT_QUEUE,
    }
}

CELERY_BEAT_SCHEDULE = {
    'test': {
        'task': 'proj.tasks.test',
        'schedule': crontab(minute='*/2')
    }
}

The task is picked up when using the default celery.beat.PersistentScheduler but not when using the celerybeatmongo.schedulers.MongoScheduler

The command I use is: celery beat -A proj -l debug -S celerybeatmongo.schedulers.MongoScheduler

The log says:

[2016-12-17 06:55:13,748: DEBUG/MainProcess] beat: Ticking with max interval->5.00 seconds
[2016-12-17 06:55:14,486: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2016-12-17 06:55:19,492: DEBUG/MainProcess] beat: Synchronizing schedule...
[2016-12-17 06:55:19,492: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2016-12-17 06:55:24,496: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.

There's no document in the mongoDB.

rs0:PRIMARY> db.schedules.find().count(); 
0 

The mongodb is my main database and the user has read-write permission.

Is there any way to fix it? Am I doing anything wrong??

[zakird]

If there's nothing in your database, then no tasks are going to be run. The library does not take care of adding things to the database for you, it reads what you have specified there.

[sudhishvnair]

I am also facing similar issue. I have 9 tasks defined in the mongo collection. However the task is only picked up once during startup. From next time onwards no task is re run at the specified interval.

{ "_id" : ObjectId("5caae53b0479fc63a8dfb7f5"), "_cls" : "PeriodicTask", "args" : [ { "lReqQueues" : [ "ocr_queue", "nlp_pdf_queue" ] } ], "enabled" : true, "interval" : { "every" : 2, "period" : "minutes" }, "kwargs" : {

},
"last_run_at" : ISODate("2019-04-08T23:30:01.669+05:30"),
"name" : "run_aws_auto_scaler",
"run_immediately" : false,
"task" : "job_man.tasks.run_aws_auto_scaler",
"total_run_count" : 14,
"type" : "defaultScheduler",
"client_id" : "0",
"scheduler_id" : "JS_50",
"updated_date" : ISODate("2019-04-04T20:49:05.068+05:30"),
"updated_by" : null

}

[Knight1997]

I am also facing similar issue. I have 9 tasks defined in the mongo collection. However the task is only picked up once during startup. From next time onwards no task is re run at the specified interval.

{ "_id" : ObjectId("5caae53b0479fc63a8dfb7f5"), "_cls" : "PeriodicTask", "args" : [ { "lReqQueues" : [ "ocr_queue", "nlp_pdf_queue" ] } ], "enabled" : true, "interval" : { "every" : 2, "period" : "minutes" }, "kwargs" : {

},
"last_run_at" : ISODate("2019-04-08T23:30:01.669+05:30"),
"name" : "run_aws_auto_scaler",
"run_immediately" : false,
"task" : "job_man.tasks.run_aws_auto_scaler",
"total_run_count" : 14,
"type" : "defaultScheduler",
"client_id" : "0",
"scheduler_id" : "JS_50",
"updated_date" : ISODate("2019-04-04T20:49:05.068+05:30"),
"updated_by" : null

}

Were you able to resolve this issue? I am facing the exact same issue