[snAPP] all seq page loading too long

[biomystery]

Comparing with metadata app. http://epigenomics.sdsc.edu:8000/metadata/index/

• there are 6,074 items in sequencing, but loading still very fast.
• Something there we can optimize.

[biomystery]

• use Ajax to load
• metadata app has applied ajax already.

[biomystery]

• django plugin
• to monitor the SQLs every behind the scene
• maybe reduce SQL calls.

[avialee]

https://github.com/jazzband/django-debug-toolbar

[biomystery]

django cache? https://docs.djangoproject.com/en/2.2/topics/cache/

https://django-datatable-view.readthedocs.io/en/latest/topics/caching.html

similar question: https://stackoverflow.com/questions/56377621/how-to-make-faster-large-data-5-000-000-in-django-with-datatables-no-django-da

another: https://www.reddit.com/r/django/comments/3hlhgq/best_way_to_render_large_data_tables_in_django/

I wrote a query engine that deals with data sets this large all the time. A number of people have 
already mentioned the need to paginate, and only pass the template the rows you are currently 
displaying.

In addition, my solution caches the query results, and runs the pagination against the cached 
results. It only re-runs the query if the cache expires. You could also store the query as session 
data, depending on your session backend, but Django's caching framework is very efficient, and 
even 18,000 rows is nothing when using memcached.

Additionally, this lets you retain the "Export CSV" button without any additional database hits.

https://stackoverflow.com/questions/57717721/large-jquery-datatables-are-slow-with-django

As mentioned in the comments, 20,000 rows is going to take a lot of time for the browser to render 
with jQuery DataTables. You may want to look into using jQuery DataTables with paginated loading. 
To do this, you need to provide your data from the Django backend using something like Django 
REST Framework.

....
jquery Datatable has Pagination option where Number of records is controlled by server side logic.

[brandonGonzalez01]

https://github.com/epigen-UCSD/epigen_ucsd_django/commit/22873ba0bba8d5d07d8a79c3f76ddaeed87e47df

[biomystery]

• wait for adding last modified time and sort on this column same as SetQC app
• check if you can isolate seq status column check into Ajax to boost loading.

[brandonGonzalez01]

https://github.com/epigen-UCSD/epigen_ucsd_django/commit/8322ee18298bdf7be7681afeeb3b5244a8ce1253 - added last modified column