Embedding a lot of data in a templates slows down Django

[gijzelaerr]

In for example our dataset detail view we show a 'number of sources' graph. This Graph is populated with data from the template. If the number of images becomes huge (like 80.000) the page becomes very slow. Usually this is caused by the database, but further inspection revealed about 70% of the time is spent on 'rendering' the data in Django templates.

Reading a bit about this issue there seem two solutions/workarounds:

• Cache template rendering (sections)
• Don't 'render' the data, but use REST api to query database with less overhead.

The first solution doesn't really apply to us, we have too few users doing too many unique request to gain from this. Also often people query their DB while populating it, which would most likely give problems where users don't see the true content.

The second solution has my preference, but depends on issue #41 which is on hold. Since this issue is just annoying but not limiting science unfortunately for now I mark this as low priority.

[gijzelaerr]

a quick test shows that template rendering Django 1.9 is in our case about 20% faster. Only problem is that there are all kind of backwards incompatible changes, so that requires some work.

[gijzelaerr]

initial work to support Django 1.9 is in this branch: https://github.com/transientskp/banana/tree/django19

but some views are failing because of an unknown reason. Also tried updating the fixtures, no go.

[gijzelaerr]

i've moved the number of sources to a new view. The dataset details view is now much quicker, that should at least make navigation around banana a lot quicker.