EVE-Tools / element43

Market, Trade and Industry Manager for EVE Online
BSD 3-Clause "New" or "Revised" License
81 stars 25 forks source link

Load all market groups from a json file instead of loading them dynamically #27

Closed jenslauterbach closed 11 years ago

jenslauterbach commented 11 years ago

Right now the performance in the "Browse Market" view is not great. The market groups are loaded dynamically on every request.

To improve performance all market groups could be "pre-compiled" in a simple JSON file. The format of the JSON file will be provided by @zweizeichen. To create the JSON I propose a Django command that can be run from the command line.

The only drawback to that solution is that the file has to be "compiled" everytime the database dump is imported.

Update 1

@gtaylor proposed to generate the file in the view and just cache it or have it generated periodically with a celery task. Either way, this would avoid the drawback described before.

gtaylor commented 11 years ago

Could even just generate it in a view, cache it in memcache, and set a very long cache header expiration time. If you've got celery set up, this could be a periodic task that makes sure we've got a "compiled" copy in memcache.

jenslauterbach commented 11 years ago

Great idea @gtaylor. Only niggle I have is that @zweizeichen and myself would prefer redis over memcache. But those are details. "Compiling" the file on the fly and caching it is the way to go!

gtaylor commented 11 years ago

You can use Redis instead, same deal.

zweizeichen commented 11 years ago

The JSON will be generated during the deploy procedure which involves collectstatic and compression of JS/(S)CSS.