search

celery / celery

Distributed Task Queue (development branch)
https://docs.celeryq.dev
Other
24.49k stars 4.65k forks source link

Celery task not getting registered in the scheduler. #6612

Open ArcD7 opened 3 years ago

ArcD7 commented 3 years ago

Checklist

Mandatory Debugging Information

Optional Debugging Information

Related Issues and Possible Duplicates

Related Issues

Possible Duplicates

Environment & Settings

5.0.5:

celery report Output:

``` software -> celery:5.0.5 (singularity) kombu:5.0.2 py:3.8.5 billiard:3.6.3.0 py-amqp:5.0.2 platform -> system:Linux arch:64bit, ELF kernel version:4.19.128-microsoft-standard imp:CPython loader -> celery.loaders.app.AppLoader settings -> transport:pyamqp results:disabled broker_url: 'amqp://******:********@localhost:5672//' deprecated_settings: None ```

Steps to Reproduce

Required Dependencies

Python Packages

pip freeze Output:

``` absl-py==0.10.0 amqp==5.0.2 aniso8601==7.0.0 argon2-cffi==20.1.0 ariadne==0.12.0 asgiref==3.2.10 astunparse==1.6.3 bcrypt==3.1.7 billiard==3.6.3.0 boto3==1.14.19 botocore==1.17.19 cachetools==4.1.1 celery==5.0.5 certifi==2020.6.20 cffi==1.14.0 chardet==3.0.4 click==7.1.2 click-didyoumean==0.0.3 click-plugins==1.1.1 click-repl==0.1.6 cycler==0.10.0 Django==3.0.7 django-amazon-ses==3.0.2 django-celery-beat==2.1.0 django-cors-headers==3.5.0 django-filter==2.2.0 django-graphql-auth==0.3.8 django-graphql-jwt==0.3.0 django-memcached==0.1.2 django-ses==1.0.1 django-storages==1.9.1 django-timezone-field==4.1.1 docutils==0.15.2 future==0.18.2 gast==0.3.3 google-auth==1.21.3 google-auth-oauthlib==0.4.1 google-pasta==0.2.0 graphene==2.1.8 graphene-django==2.9.0 graphene-file-upload==1.2.2 graphql-core==2.3.2 graphql-relay==2.0.1 grpcio==1.32.0 h5py==2.10.0 idna==2.10 jmespath==0.10.0 joblib==0.16.0 Keras==2.4.3 Keras-Preprocessing==1.1.2 kiwisolver==1.2.0 kombu==5.0.2 Markdown==3.2.2 matplotlib==3.3.2 nltk==3.5 numpy==1.18.5 oauthlib==3.1.0 opt-einsum==3.3.0 Pillow==7.2.0 promise==2.3 prompt-toolkit==3.0.10 protobuf==3.13.0 psycopg2-binary==2.8.5 pyasn1==0.4.8 pyasn1-modules==0.2.8 pycparser==2.20 PyJWT==1.7.1 pyparsing==2.4.7 python-crontab==2.5.1 python-dateutil==2.8.1 python-memcached==1.59 pytz==2020.1 PyYAML==5.3.1 regex==2020.7.14 requests==2.24.0 requests-oauthlib==1.3.0 rsa==4.6 Rx==1.6.1 s3transfer==0.3.3 scipy==1.4.1 singledispatch==3.4.0.3 six==1.15.0 sqlparse==0.3.1 starlette==0.13.8 text-unidecode==1.3 tqdm==4.49.0 typing-extensions==3.7.4.3 Unidecode==1.1.1 urllib3==1.25.9 vine==5.0.0 wcwidth==0.2.5 Werkzeug==1.0.1 wordcloud==1.8.0 wrapt==1.12.1 ```

Other Dependencies

N/A

Minimally Reproducible Test Case

` from celery import Celery from celery.schedules import crontab app = Celery("tasks", broker="pyamqp://username:password@localhost//") def clean_time(time): time = time.get("created_at") hh = time.strftime("%H") mm = time.strftime("%M") ss = time.strftime("%S") dd = time.strftime("%d") mnth = time.strftime("%m") yy = time.strftime("%Y") return hh, mm, ss, dd, mnth, yy @app.on_after_configure.connect def start_time(time, **kwargs): hh, mm, ss, dd, mnth, yy = clean_time(time) print(hh, mm, ss, dd, mnth, yy) app.add_periodic_task( crontab( hour="{}".format(hh), minute="{}".format(mm), day_of_month="{}".format(dd), month_of_year="{}".format(mnth), ), test.s("hello"), ) @app.task def test(arg): print(arg) `

Expected Behavior

Actual Behavior

celery beat v5.0.5 (singularity) is starting. __ - ... __ - _ LocalTime -> 2021-01-22 19:37:35 Configuration -> . broker -> amqp://archit:**@localhost:5672// . loader -> celery.loaders.app.AppLoader . scheduler -> celery.beat.PersistentScheduler . db -> celerybeat-schedule . logfile -> [stderr]@%DEBUG . maxinterval -> 5.00 minutes (300s) [2021-01-22 19:37:35,166: DEBUG/MainProcess] Setting default socket timeout to 30 [2021-01-22 19:37:35,167: INFO/MainProcess] beat: Starting... [2021-01-22 19:37:35,207: DEBUG/MainProcess] Current schedule: <ScheduleEntry: celery.backend_cleanup celery.backend_cleanup() <crontab: 0 4 * * * (m/h/d/dM/MY)> [2021-01-22 19:37:35,207: DEBUG/MainProcess] beat: Ticking with max interval->5.00 minutes [2021-01-22 19:37:35,208: DEBUG/MainProcess] beat: Waking up in 5.00 minutes. Thank you in advance.

lewisemm commented 3 years ago

@ArcD7 @auvipy

If the code provided as the minimally reproducible test case was written as intended, this may not be an issue with Celery.

The start_time function should be using its first argument to access add_periodic_task method. So instead of ...


@app.on_after_configure.connect
def start_time(time, **kwargs):
    hh, mm, ss, dd, mnth, yy = clean_time(time)
    print(hh, mm, ss, dd, mnth, yy)
    app.add_periodic_task(
        crontab(
            hour="{}".format(hh),
            minute="{}".format(mm),
            day_of_month="{}".format(dd),
            month_of_year="{}".format(mnth),
        ),
        test.s("hello"),
    )

try this instead.


@app.on_after_configure.connect
def start_time(sender, **kwargs):
    hh, mm, ss, dd, mnth, yy = clean_time(time)
    print(hh, mm, ss, dd, mnth, yy)
    sender.add_periodic_task(
        crontab(
            hour="{}".format(hh),
            minute="{}".format(mm),
            day_of_month="{}".format(dd),
            month_of_year="{}".format(mnth),
        ),
        test.s("hello"),
    )
ArcD7 commented 3 years ago

I've tried this also, but still my task would not get registered.

thedrow commented 3 years ago

Is there an exception? How do you get the time argument?

ArcD7 commented 3 years ago

There are no errors. I get the time argument from the Django Views file.

On Sun, 7 Mar 2021 at 9:09 PM, Omer Katz notifications@github.com wrote:

Is there an exception? How do you get the time argument?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/celery/celery/issues/6612#issuecomment-792298315, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALFETZTKAWF3XWNHV2JK2G3TCOM3XANCNFSM4WORN76A .

thedrow commented 3 years ago

Can you please submit a full test case?

ArcD7 commented 3 years ago

This is the basic schedule function that I was using.


@app.on_after_configure.connect
def start_time(time, **kwargs):

    hh, mm, ss, dd, mnth, yy = clean_time(time)
    print(hh, mm, ss, dd, mnth, yy)
    app.add_periodic_task(
        crontab(
            hour="{}".format(hh),
            minute="{}".format(mm),
            day_of_month="{}".format(dd),
            month_of_year="{}".format(mnth),
        ),
        test.s("hello"),
    )

There is a function in the Django views file which calls the start time function and passes the time argument which is a nothing but a datetime object which gets further segregated into its components using another function clean time.

P.S: When I was trying out this code various methods were tried like the kwargs with the sender one(which did not work) as specified by @lewisemm but there might be a case that I may have passed the kwarg argument in a wrong way!

manavchawla2012 commented 3 years ago

I am getting exact same issue {"exc_type": "NotRegistered", "exc_message": ["summary"], "exc_module": "celery.exceptions"}

afridsyed326 commented 1 year ago

Running into same issue.

{"exc_type": "NotRegistered", "exc_message": ["my_app.tasks.test"], "exc_module": "celery.exceptions"}

Below is my scheduler code

CELERY_BEAT_SCHEDULE = { # scheduler configuration 
    'credit_matching_bonus' : {  # whatever the name you want 
        'task': 'my_app.tasks.test', # name of task with path
        'schedule': crontab(minute=45),
    }
}

This is how i am registering the task

# tasks.py
from celery import shared_task
@shared_task
def test():
      ** logic **

Below is the log 

[2023-08-03 12:38:20,157: INFO/MainProcess] beat: Starting...
[2023-08-03 12:38:45,746: INFO/MainProcess] beat: Starting...
[2023-08-03 12:45:00,066: INFO/MainProcess] Scheduler: Sending due task test (my_app.tasks.test)

Any suggesstions would be truly helpful and sorry i am new to this.