Koed00 / django-q

A multiprocessing distributed task queue for Django
https://django-q.readthedocs.org
MIT License
1.83k stars 285 forks source link

Scheduled tasks failed when deploying app on heroku #645

Closed mikolajnowak97 closed 2 years ago

mikolajnowak97 commented 2 years ago

Hi, I am trying to deploy my django app on heroku with django q schedules. I am using django q async tasks and I haven't encountered any problems. Now I created some schedules but I get:

2022-01-24T00:47:16.931356+00:00 app[worker.1]: 00:47:16 [Q] ERROR Failed [mexico-hawaii-idaho-early] - 'NoneType' object is not callable : Traceback (most recent call last): 2022-01-24T00:47:16.931357+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.9/site-packages/django_q/cluster.py", line 432, in worker 2022-01-24T00:47:16.931358+00:00 app[worker.1]: res = f(*task["args"], **task["kwargs"]) 2022-01-24T00:47:16.931360+00:00 app[worker.1]: TypeError: 'NoneType' object is not callable

When I deploy app locally everything works fine. Problem starts when app is running on heroku. I checked my version of django_q and version declared in requirements and it's all right. Where can I find solution for this? python: 3.9.6 django: 3.2.7 django_q: 1.3.9

GDay commented 2 years ago

I am having the same issue. I have tried to downgrade the python runtime to 3.7 and 3.8.

With 3.8, I get the same error. With 3.7, I still get the same error and also a new one (unsupported pickle protocol).

I have also tried downgrading the Django-q version, but that didn't work either. I am not entirely sure yet what the issue is. I will update when I know more.

mikolajnowak97 commented 2 years ago

I have tried different combinations of creating scheduled tasks.

First, directly from code with the schedule() function or the Schedule model defined in docs. Then I tried to create them via admin pages and provide different combinations of args and kwargs, but all attempts ended with the same error.

Currently I am using APScheduler, but I'll keep looking for solution and share it if I find one.

jyoost commented 2 years ago

Check if the library for pickle (system library in Docker container) is installed. Smaller images sometimes don't install all libraries.

GDay commented 2 years ago

Check if the library for pickle (system library in Docker container) is installed. Smaller images sometimes don't install all libraries.

Thanks! Good point, that's will probably solve that pickle error.

As for the main issue; I think I figured it out. For me, I had moved a few tasks and now the funct field in the Schedule model wasn't correct anymore. Therefore it couldn't pick up the function and returned a NoneType object. My bad :man_facepalming:

So, if it's anything like that for you @mikolajnowak97 then definitely check if the records are correct in your Schedule model.

>>> from django_q.models import Schedule
>>> for schedule in Schedule.objects.all():
...     print(schedule.func)

And also remove any that aren't in use anymore.

mikolajnowak97 commented 2 years ago

Hi, thank you ! I didnt have to install pickle library. My problem was a bit strange and I don't really understand it.

I had all Schedule records in one directory/file that was working on my local machine. When I deploy app to heroku that configuration wasnt correct (because of invalid func path as mentioned by @GDay). Then I decided to use APScheduler and all was working good.

So I when @GDay mentioned his solution I recreated my old file with Schedule model initialization and all is working fine. I dont understand why when I didnt change anything 🤷 Also I deleted APScheduler from my project (from requirements.txt and django installed aps). With new configuration I deployed new project to heroku and everything is working greate.

But even I deleted whole APScheduler from my project, on production environment of my app, I can still see 'DJANGO APSCHEDULER' section in ADMIN page. I dont know why it still there, when I deleted it already, and on local machine I dont see it. It looks like heroku environments refreshing with some of a delay (if it is possible 🤷)

Anyway, thank you all very much for your help

881997 commented 2 years ago

I have the same error because I use a relative reference like: from moduleA import func async_task('func') The right way: async_task('myapp.moduleA.func')

issacmark commented 2 years ago

I have the same error because I use a relative reference like: from moduleA import func async_task('func') The right way: async_task('myapp.moduleA.func')

i am using async_task('myapp.moduleA.func') but still getting: res = f(*task["args"], **task["kwargs"]) TypeError: 'NoneType' object is not callable

when i check the admin page for failed task, args is not None