search

MI-DPLA / combine

Combine /kämˌbīn/ - Metadata Aggregator Platform
MIT License
26 stars 11 forks source link

Investigate Django-Datatables connector reponsiveness #276

Open ghukill opened 6 years ago

ghukill commented 6 years ago

The Django-Datatables connector uses Django's ORM to handle datatables requests. Switching out MySQL for Mongo helped considerably for the responsiveness here, but they are still somewhat slow for large Jobs (1 million+).

However, on the command line, using slices and offsets with pymongo or mongoengine, it's possible to very quickly move through results sets. Wondering if the Django ORM is issuing uneeded .count() commands for the connector to show total and limits?

It would be a shame to lose some of the built-in functionality of the Django datatables connector, but it's also not a well maintained library, and might be a liability already.

antmoth commented 5 years ago

Django ORM does have two different "how big is this table" methods.