Closed dpapathanasiou closed 10 years ago
Using "latest()" like this in the ORM is efficient (especially now that the rest of the query uses Q models for the pending/success "or" condition rather than creating a sub query, which is what the '__in=[]' was doing), so the right place to add the time frame limitation is in the "query_mongo()" function, which is another beast entirely...
Not about query efficiency, but more about when to trigger exports (which unleashes the mongo beast). We can walk through it tomorrow or Monday. On Apr 24, 2014 6:14 PM, "Denis Papathanasiou" notifications@github.com wrote:
Using "latest()" like this in the ORM is efficient (especially now that the rest of the query uses Q models for the pending/success "or" condition rather than creating a sub query, which is what the '__in=[]' was doing), so the right place to add the time frame limitation is in the "query_mongo()" function, which is another beast entirely...
Reply to this email directly or view it on GitHubhttps://github.com/SEL-Columbia/formhub/pull/1322#issuecomment-41339245 .
Right, that's what I meant about changing "query_mongo()" -- setting a date range or limit here, on the ORM query won't do that.
Changing "query_mongo()" to do just the diffs is also not that easy, for reasons too involved to explain here.
Made some improvements to how the export logic checks the relational db/django ORM:
This logic flow is still very ugly: the exports_outdated() function really should not be a classmethod of Export, but doing it incrementally like this for now, until can do a more thorough overhaul later.