Closed levinas closed 9 years ago
OperationFailure: database error: too much data for sort() with no index. add an index or specify a smaller limit
I think the current implementation does something dumb and queries all user jobs first and limits the records in Python space. We need to limit the query in the mongo call instead.
Oh wait.. this is a sort problem. I guess mongo needs to sort before returning the latest jobs. We probably need to index.
ensureIndex !
The query is:
jobs.find({'ARASTUSER':user}).sort('job_id', 1)
For the find command:
db.jobs.ensureIndex({'ARASTUSER': 1});
For the sort command:
db.jobs.ensureIndex({'job_id': 1});
That does it? Could you create a patch?
On Feb 23, 2015, at 5:52 PM, Sébastien Boisvert notifications@github.com wrote:
ensureIndex !
The query is:
jobs.find({'ARASTUSER':user}).sort('job_id', 1)
For the find command:
db.jobs.ensureIndex({'ARASTUSER': 1});
For the sort command:
db.jobs.ensureIndex({'job_id': 1});
— Reply to this email directly or view it on GitHub https://github.com/kbase/assembly/issues/299#issuecomment-75655622.
These can be run from the mongo shell. But we should also add some code that create these indexes in the controller daemon on startup I suppose.
The jenkins test user has submitted enough jobs (~2000) to trigger this limit in mongodb sort: