My scheduler script is running every one minute scheduler.cron("* * * * *", func=my_func, queue_name=QueueName.My_QUEUE_NAME)
After running four to five minutes rqscheduler stopped and throws an error
19:23:59 Registering death
Traceback (most recent call last):
File "/opt/edugem/apps/psh_core/venv/lib/python3.7/site-packages/rq/job.py", line 53, in unpickle
obj = loads(pickled_string)
_pickle.UnpicklingError: pickle data was truncated
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/edugem/apps/psh_core/venv/bin/rqscheduler", line 10, in <module>
sys.exit(main())
File "/opt/edugem/apps/psh_core/venv/lib/python3.7/site-packages/rq_scheduler/scripts/rqscheduler.py", line 61, in main
scheduler.run(burst=args.burst)
File "/opt/edugem/apps/psh_core/venv/lib/python3.7/site-packages/rq_scheduler/scheduler.py", line 413, in run
self.enqueue_jobs()
File "/opt/edugem/apps/psh_core/venv/lib/python3.7/site-packages/rq_scheduler/scheduler.py", line 391, in enqueue_jobs
for job in jobs:
File "/opt/edugem/apps/psh_core/venv/lib/python3.7/site-packages/rq_scheduler/scheduler.py", line 322, in get_jobs
job = self.job_class.fetch(job_id, connection=self.connection)
File "/opt/edugem/apps/psh_core/venv/lib/python3.7/site-packages/rq/job.py", line 288, in fetch
job.refresh()
File "/opt/edugem/apps/psh_core/venv/lib/python3.7/site-packages/rq/job.py", line 462, in refresh
self.restore(data)
File "/opt/edugem/apps/psh_core/venv/lib/python3.7/site-packages/rq/job.py", line 435, in restore
self._result = unpickle(obj.get('result')) if obj.get('result') else None # noqa
File "/opt/edugem/apps/psh_core/venv/lib/python3.7/site-packages/rq/job.py", line 55, in unpickle
raise UnpickleError('Could not unpickle', pickled_string, e)
rq.exceptions.UnpickleError: ('Could not unpickle', UnpicklingError('pickle data was truncated'))
Now It only can able to re-run rqscheduler after clearing all my redis records. But need a genuine solution.
I am using ubuntu 19.04, python 3.6, rq-scheduler 0.9.1, and Django 2.2.4.
I need to register the scheduler and for that, I am using django-extension.
From django-extension, I am adding data into my queue and running queue from rq-worker.
step 1: starting django server step 2: running
rqscheduler
step 3: runningpython3 manage.py runscript scheduler
step 4: running specific workerrqworker MY_WORKER_NAME
My scheduler script is running every one minute
scheduler.cron("* * * * *", func=my_func, queue_name=QueueName.My_QUEUE_NAME)
After running four to five minutes
rqscheduler
stopped and throws an errorNow It only can able to re-run
rqscheduler
after clearing all my redis records. But need a genuine solution.