Investigate the practice of many "store" workers accessing the DB simultaneously

[chenwardT]

Sometimes IntegrityErrors will get thrown if the number of workers in the RabbitMQ queue store is greater than 1.

It could be that this exception can be safely ignored.

Possible solutions:

• Transactions in the right places might make this go away. Beware idling in transactions.
• Writes to the DB could be pooled and done in bulk after ensuring none of them "overlap" or conflict. This can only be done in certain places, as a user requesting an update on their page must take priority over efficiency in other regards.

[chenwardT]

Sent 100 summoner IDs to bulk.get_leagues_for_summoner_ids, worked w/o error (default worker config: 1 of each kind of worker, I think).

Sent another 100 after upping workers to 10/10/100 (default / match_ids / store) and got this kind of exception multiple times:

[2015-10-30 20:51:33,363: ERROR/MainProcess] Task lol_stats2.celery.store_get_league[14c91ccf-f8f0-4081-b3e9-ae8dc47e21ea] raised unexpected: IntegrityError('duplicate key value violates unique constraint "leagues_leagueentry_player_or_
Traceback (most recent call last):                                                                                                                                                                                                          
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/celery/app/trace.py", line 240, in trace_task                                                                                                                    
    R = retval = fun(*args, **kwargs)                                                                                                                                                                                                       
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/celery/app/trace.py", line 438, in __protected_call__                                                                                                            
    return self.run(*args, **kwargs)                                                                                                                                                                                                        
  File "/home/chen/python-projects/lol_stats2/lol_stats2/lol_stats2/celery.py", line 311, in store_get_league                                                                                                                               
    League.objects.create_or_update_league(league, region)                                                                                                                                                                                  
  File "/home/chen/python-projects/lol_stats2/lol_stats2/leagues/models.py", line 19, in create_or_update_league                                                                                                                            
    return self.update_league(league, attrs, region)                                                                                                                                                                                        
  File "/home/chen/python-projects/lol_stats2/lol_stats2/leagues/models.py", line 34, in update_league                                                                                                                                      
    league.leagueentry_set.create_entries(attrs)                                                                                                                                                                                            
  File "/home/chen/python-projects/lol_stats2/lol_stats2/leagues/models.py", line 119, in create_entries                                                                                                                                    
    self.create_entry(entry)                                                                                                                                                                                                                
  File "/home/chen/python-projects/lol_stats2/lol_stats2/leagues/models.py", line 95, in create_entry                                                                                                                                       
    losses=attrs['losses'])                                                                                                                                                                                                                 
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/models/fields/related.py", line 750, in create                                                                                                         
    return super(RelatedManager, self.db_manager(db)).create(**kwargs)                                                                                                                                                                      
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/models/manager.py", line 127, in manager_method                                                                                                        
    return getattr(self.get_queryset(), name)(*args, **kwargs)                                                                                                                                                                              
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/models/query.py", line 348, in create                                                                                                                  
    obj.save(force_insert=True, using=self.db)                                                                                                                                                                                              
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/models/base.py", line 710, in save                                                                                                                     
    force_update=force_update, update_fields=update_fields)                                                                                                                                                                                 
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/models/base.py", line 738, in save_base                                                                                                                
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)                                                                                                                                                  
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/models/base.py", line 822, in _save_table                                                                                                              
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)                                                                                                                                                              
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/models/base.py", line 861, in _do_insert                                                                                                               
    using=using, raw=raw)                                                                                                                                                                                                                   
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/models/manager.py", line 127, in manager_method                                                                                                        
    return getattr(self.get_queryset(), name)(*args, **kwargs)                                                                                                                                                                              
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/models/query.py", line 920, in _insert                                                                                                                 
    return query.get_compiler(using=using).execute_sql(return_id)                                                                                                                                                                           
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 971, in execute_sql                                                                                                      
    cursor.execute(sql, params)                                                                                                                                                                                                             
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/backends/utils.py", line 79, in execute                                                                                                                
    return super(CursorDebugWrapper, self).execute(sql, params)                                                                                                                                                                             
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute                                                                                                                
    return self.cursor.execute(sql, params)                                                                                                                                                                                                 
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/utils.py", line 97, in __exit__                                                                                                                        
    six.reraise(dj_exc_type, dj_exc_value, traceback)                                                                                                                                                                                       
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/utils/six.py", line 658, in reraise                                                                                                                       
    raise value.with_traceback(tb)                                                                                                                                                                                                          
  File "/home/chen/.virtualenvs/lol_stats2_dev/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute                                                                                                                
    return self.cursor.execute(sql, params)                                                                                                                                                                                                 
django.db.utils.IntegrityError: duplicate key value violates unique constraint "leagues_leagueentry_player_or_team_id_2b411fd0cb827995_uniq"                                                                                                
DETAIL:  Key (player_or_team_id, league_id)=(19891056, 9780) already exists.

This was NOT fixed by adding transaction.atomic to create_or_update_league.