search

chenwardT / lol_stats2

Website to perform match aggregation and analysis for Riot Games' League of Legends.
2 stars 0 forks source link

WorkerLostError w/exit code 0 in riot_api task #21

Closed chenwardT closed 8 years ago

chenwardT commented 8 years ago

Searches on the subject point to how exceptions are raised, esp. re: using BaseException instead of specific Exceptions. See http://stackoverflow.com/questions/27612584/debugging-celery-workerlosterror-with-exitcode-zero-django-1-5-5-celery-3-1-8

Example output:

[2015-11-25 14:00:17,649: INFO/MainProcess] Received task: lol_stats2.celery.riot_api[5c259aed-53a9-48d0-b48b-ecc7607184e6]                                                                                                                
[2015-11-25 14:00:17,652: DEBUG/Worker-2] kwargs: {'begin_time': None, 'method': 'get_match_list', 'end_index': None, 'champion_ids': None, 'summoner_id': 23792880, 'region': 'NA', 'season': None, 'begin_index': None, 'ranked_queues': 
[2015-11-25 14:00:17,652] DEBUG [lol_stats2.celery:76] riot_api: kwargs: {'begin_time': None, 'method': 'get_match_list', 'end_index': None, 'champion_ids': None, 'summoner_id': 23792880, 'region': 'NA', 'season': None, 'begin_index': 
[2015-11-25 14:00:17,656: INFO/Worker-2] Starting new HTTPS connection (1): na.api.pvp.net                                                                                                                                                 
[2015-11-25 14:00:18,158: WARNING/Worker-2] --- Logging error ---                                                                                                                                                                          
[2015-11-25 14:00:18,177: ERROR/MainProcess] Task lol_stats2.celery.riot_api[5c259aed-53a9-48d0-b48b-ecc7607184e6] raised unexpected: WorkerLostError('Worker exited prematurely: exitcode 0.',)                                           
Traceback (most recent call last):                                                                                                                                                                                                         
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/billiard/pool.py", line 1175, in mark_as_worker_lost                                                                                                            
    human_status(exitcode)),                                                                                                                                                                                                               
mbilliard.exceptions.WorkerLostError: Worker exited prematurely: exitcode 0.                                                                                                                                                                
l[2015-11-25 14:00:18,651] ERROR [django.request:256] handle_uncaught_exception: Internal Server Error: /refresh/                                                                                                                           
Traceback (most recent call last):                                                                                                                                                                                                         
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/core/handlers/base.py", line 132, in get_response                                                                                                        
    response = wrapped_callback(request, *callback_args, **callback_kwargs)                                                                                                                                                                
  File "/home/chen/python-projects/lol_stats2/lol_stats2/summoners/views.py", line 69, in refresh                                                                                                                                          
    task_ids = ss.full_query()                                                                                                                                                                                                             
  File "/home/chen/python-projects/lol_stats2/lol_stats2/cache/summoners.py", line 240, in full_query                                                                                                                                      
    match_job = self._query_match_history()                                                                                                                                                                                                
  File "/home/chen/python-projects/lol_stats2/lol_stats2/cache/summoners.py", line 162, in _query_match_history                                                                                                                            
    region=self.summoner.region)                                                                                                                                                                                                           
  File "/home/chen/python-projects/lol_stats2/lol_stats2/riot_api/wrapper.py", line 222, in get_match_list                                                                                                                                 
    store_get_match.s()) for match_id in get_ids_chain().get())()                                                                                                                                                                          
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/celery/result.py", line 169, in get                                                                                                                             
    no_ack=no_ack,                                                                                                                                                                                                                         
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/celery/backends/base.py", line 221, in wait_for                                                                                                                 
    on_interval()                                                                                                                                                                                                                          
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/celery/result.py", line 181, in _maybe_reraise_parent_error                                                                                                     
    node.maybe_reraise()                                                                                                                                                                                                                   
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/celery/result.py", line 271, in maybe_reraise                                                                                                                   
    raise self.result                                                                                                                                                                                                                      
billiard.exceptions.WorkerLostError: Worker exited prematurely: exitcode 0.
chenwardT commented 8 years ago

Can also occur when getting results of a celery chain.

Note: This was first noticed after changing riot_api task to use acks_late.

In [1]: sums = Summoner.objects.filter(region='NA')[10000:20000]

In [2]: from riot_api.wrapper import RiotAPI

In [3]: [RiotAPI.get_match_list(s.summoner_id, region='NA', max_matches=1) for s in sums]
---------------------------------------------------------------------------
WorkerLostError                           Traceback (most recent call last)
<ipython-input-3-ea52815032dc> in <module>()
----> 1 [RiotAPI.get_match_list(s.summoner_id, region='NA', max_matches=1) for s in sums]

<ipython-input-3-ea52815032dc> in <listcomp>(.0)
----> 1 [RiotAPI.get_match_list(s.summoner_id, region='NA', max_matches=1) for s in sums]

/home/chen/python-projects/lol_stats2/lol_stats2/riot_api/wrapper.py in get_match_list(summoner_id, region, champion_ids, ranked_queues, season, begin_time, end_time, begin_index, end_index, max_matches)
    220             return group(chain(RiotAPI.get_match.s(match_id=match_id, region=region),
    221                                riot_api.s(),
--> 222                                store_get_match.s()) for match_id in get_ids_chain().get())()
    223
    224     @app.task

/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/celery/result.py in get(self, timeout, propagate, interval, no_ack, follow_parents, EXCEPTION_STATES, PROPAGATE_STATES)
    167             interval=interval,
    168             on_interval=on_interval,
--> 169             no_ack=no_ack,
    170         )
    171         if meta:

/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/celery/backends/base.py in wait_for(self, task_id, timeout, interval, no_ack, on_interval)
    219                 return meta
    220             if on_interval:
--> 221                 on_interval()
    222             # avoid hammering the CPU checking status.
    223             time.sleep(interval)

/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/celery/result.py in _maybe_reraise_parent_error(self)
    179     def _maybe_reraise_parent_error(self):
    180         for node in reversed(list(self._parents())):
--> 181             node.maybe_reraise()
    182
    183     def _parents(self):

/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/celery/result.py in maybe_reraise(self)
    269     def maybe_reraise(self):
    270         if self.state in states.PROPAGATE_STATES:
--> 271             raise self.result
    272
    273     def build_graph(self, intermediate=False, formatter=None):

WorkerLostError: Worker exited prematurely: exitcode 0.
chenwardT commented 8 years ago

Error sourced to LoLException being unhashable. See https://github.com/pseudonym117/Riot-Watcher/issues/46

chenwardT commented 8 years ago

Pending something better by author, fixed by: https://github.com/chenwardT/lol_stats2/commit/f03822e4999ce019b26c38c8335fd65ab72107f8