search

codeeu / coding-events

A Django app for adding EU Code Week events and showing them on a map.
http://events.codeweek.eu
MIT License
17 stars 36 forks source link

Optimize the events map or find another approach for it #444

Open mitio opened 7 years ago

mitio commented 7 years ago

I've temporary disabled the map in 5041a41 as it's too slow and practically incapacitates the site.

There are over 6000 events for this year's edition and they load on each homepage hit. When the homepage is loaded, a separate asynchronous (AJAX) request is made to the server to fetch a list of all the events using this URL: http://events.codeweek.eu/api/event/list/?format=json. It returns 6000+ records along with their ID, coordinates, title, URL name, short description and image URL. That’s a lot of data. It’s over 2 MB of text, for just that request. It’s cached in general, but the server seems to be unable to even build the cache. I managed to get it loading only when I allowed access only for my IP to the server. Then it was slow at first (~5-7s to load the homepage and then another 10-12s to load the list of events), then after a few requests it managed to cache the results and it became faster (loading the page in matter of a few seconds total).

The concept here needs to change. I can think of the following quickish workarounds currently:

  1. Aggregate the events each few minutes in a periodic server job and then keep these aggregated events in a file or in the DB and the application should use this file instead of loading all that information every time. The map is dynamic, however, and changes the grouping of the events when one zooms in or out. That means that a few cached files should be kept, for each zoom level, or at least for a few of the most zoomed out levels, as the most events are shown there.
  2. Another quackish approach would be to just load the ID and location of the event and when clicked on the marker, to load that event’s page. We currently show a popup with some event information when clicked on the marker. This would make the resulting JSON data with the events about 5-6 times smaller, from over 2 MB to about 370 KB. Still not ideal, but better, faster to generate, faster to cache, faster to deliver, faster to process in the browser.

I think option 2 is easier to implement. Both options would preserve most of the existing functionality.

Optimizing the map is not enough, though, more parts of the site need optimizations as well.

alessandrobogliolo commented 7 years ago

Both options are valuable and should be implemented. What is surprising is that there were no benefits in spite of the significant the upgrade of the droplet. Also, I'm surprised that the map became too heavy to be loaded at all in a very short time, without a huge increment of mapped events (numbers so far are similar to thos of last year).

Then we need also to think at event mapping and approval. Those functionalities are probably more critical and more urgent than map rendering, and they cannot be cached. Last time I checked the DB there were more than 1500 pending events. We should probably operate directly on the DB and run some bulk processing to approve them and reject only fake events and duplicates.

Finally, I wonder why the events.codeweek.eu site is still so slow, even without map.

2016-10-12 23:25 GMT+02:00 Dimitar Dimitrov notifications@github.com:

I've temporary disabled the map in 5041a41 https://github.com/codeeu/coding-events/commit/5041a41d3ce9b20824977ad97bbf2e44f024760d as it's too slow and practically incapacitates the site.

There are over 6000 events for this year's edition and they load on each homepage hit. When the homepage is loaded, a separate asynchronous (AJAX) request is made to the server to fetch a list of all the events using this URL: http://events.codeweek.eu/api/event/list/?format=json. It returns 6000+ records along with their ID, coordinates, title, URL name, short description and image URL. That’s a lot of data. It’s over 2 MB of text, for just that request. It’s cached in general, but the server seems to be unable to even build the cache. I managed to get it loading only when I allowed access only for my IP to the server. Then it was slow at first (~5-7s to load the homepage and then another 10-12s to load the list of events), then after a few requests it managed to cache the results and it became faster (loading the page in matter of a few seconds total).

The concept here needs to change. I can think of the following quickish workarounds currently:

1.

Aggregate the events each few minutes in a periodic server job and then keep these aggregated events in a file or in the DB and the application should use this file instead of loading all that information every time. The map is dynamic, however, and changes the grouping of the events when one zooms in or out. That means that a few cached files should be kept, for each zoom level, or at least for a few of the most zoomed out levels, as the most events are shown there. 2.

Another quackish approach would be to just load the ID and location of the event and when clicked on the marker, to load that event’s page. We currently show a popup with some event information when clicked on the marker. This would make the resulting JSON data with the events about 5-6 times smaller, from over 2 MB to about 370 KB. Still not ideal, but better, faster to generate, faster to cache, faster to deliver, faster to process in the browser.

I think option 2 is easier to implement. Both options would preserve most of the existing functionality.

Optimizing the map is not enough, though, more parts of the site need optimizations as well.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/codeeu/coding-events/issues/444, or mute the thread https://github.com/notifications/unsubscribe-auth/AIPCH_N9gSCtS7dg5GNozf8D6eBO8613ks5qzVBhgaJpZM4KVPPu .