komuw / wiji

Wiji is an asyncio distributed task processor/queue.
MIT License
4 stars 1 forks source link

task.task_options is stale #38

Closed komuw closed 5 years ago

komuw commented 5 years ago
(Pdb) str(self.the_task)
"{'task_name': 'CPUTask', 
'the_broker': <wijisqs.broker.SqsBroker object at 0x10d8b8630>,
 'queue_name': 'CPUTaskQueue',
 'chain': None,
 'task_options': {'eta': '2019-03-23T15:27:32.546361+00:00',
 'current_retries': 0, 
'max_retries': 0, 
'log_id': '',
 'hook_metadata': '', 
'task_id': '', 
'drain_duration': 10.0, 'args': (), 'kwargs': {}}}"

also at this option: https://github.com/komuw/wiji/blob/e02ee57590c7dc29bf7910861884feb33968247e/wiji/worker.py#L336-L339

(Pdb) str(self.the_task.task_options)
"{'eta': '2019-03-23T15:27:32.546361+00:00',
 'current_retries': 0,
 'max_retries': 0, 
'log_id': '',
 'hook_metadata': '',
 'task_id': '',
 'drain_duration': 10.0, 'args': (), 'kwargs': {}}"
(Pdb)
komuw commented 5 years ago

maybe we should stop handing over class instances to other classes

komuw commented 5 years ago
{'event': 'wiji.Task._notify_hook', 'stage': 'end', 'state': 'task hook error', 'error': "'CPUTask' object has no attribute 'task_options'", 'task_name': 'CPUTask', 'queue_name': 'CPUTaskQueue', 'worker_id': '0E7EN33UZ5UFMTD4H', 'process_id': 34323}
Traceback (most recent call last):
  File "/Users/home/wiji-benchmarks/.venv/lib/python3.7/site-packages/wiji/task.py", line 392, in _notify_hook
    task_id=self.task_options.task_id,
AttributeError: 'CPUTask' object has no attribute 'task_options'
{'event': 'wiji.Worker.consume_tasks', 'stage': 'end', 'state': 'dequeue tasks failed. sleeping for 0.5minutes', 'dequeue_retry_count': 1, 'error': '', 'task_name': '_watchdogTask', 'queue_name': '__WatchDogTaskQueue__', 'worker_id': '8A37UD9AY9914Q976', 'process_id': 34323}
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 568, in run_until_complete
    return future.result()
  File "/Users/home/wiji-benchmarks/.venv/lib/python3.7/site-packages/cli/cli.py", line 126, in async_main
    await gather_tasks
  File "/Users/home/wiji-benchmarks/.venv/lib/python3.7/site-packages/wiji/worker.py", line 335, in consume_tasks
    await self.run_task(*task_args, **task_kwargs)
  File "/Users/home/wiji-benchmarks/.venv/lib/python3.7/site-packages/wiji/worker.py", line 171, in run_task
    state=task.TaskState.EXECUTING, hook_metadata=self.the_task.task_options.hook_metadata
AttributeError: 'CPUTask' object has no attribute 'task_options'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/home/wiji-benchmarks/.venv/lib/python3.7/site-packages/wiji/worker.py", line 283, in consume_tasks
    queue_name=self.the_task.queue_name
  File "/Users/home/wiji-benchmarks/.venv/lib/python3.7/site-packages/wiji/broker.py", line 114, in dequeue
    await asyncio.sleep(5)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/tasks.py", line 564, in sleep
    return await future
concurrent.futures._base.CancelledError