search

chenwardT / lol_stats2

Website to perform match aggregation and analysis for Riot Games' League of Legends.
2 stars 0 forks source link

First time full queries of summoners not in the DB are slow #19

Closed chenwardT closed 8 years ago

chenwardT commented 8 years ago

ss.first_time_query() for a summoner that doesn't exist (or have any of their last 10 games in the database) takes ~20 seconds w/current hardware on LGZ machine.

Celery worker settings: default worker concurrency: 10 match_ids worker concurrency: 1 (no effect changing to 10) store worker concurrency: 10

get_matches_from_ids: max_matches: 10 recent_first: True

The longest running tasks are store_get_match and take anywhere from 3.6s to 5.3s (albeit on some aging drives). Since we get 10 matches, this task occurs 10 times, so this is a good starting point for cutting down initial query time.

chenwardT commented 8 years ago

Reproducing the setup on a machine w/similar specs except with an SSD lowered store_get_match jobs to 0.13 - 0.14 second range.

riot_api jobs are now the longest running tasks by far, but store_get_match could be improved, perhaps pending any schema changes in light of this more realistic benchmark.

chenwardT commented 8 years ago

store_get_match jobs are now down to 0.10 - 0.11 seconds after the switch to bulk ops for many inserts.

store_get_league occurs in around 0.07 seconds with similar bulk insertion optimizations.

store_get_summoners has always been fast, but for completeness, occurs in 0.01 seconds.