search

WIPACrepo / file_catalog

Store file metadata information in a file catalog
MIT License
1 stars 4 forks source link

PyMongo: `skip` and `limit` are ignored by `__getitem__` #6

Closed JadonKrys closed 7 years ago

JadonKrys commented 7 years ago

The limit and skip parameter of the Mongo.find_files() method are ignored because __getitem__ ignores them, see http://api.mongodb.com/python/current/api/pymongo/cursor.html#pymongo.cursor.Cursor.__getitem__

As the start parameter has been implemented as a list slice, it is already supported. However, the limit parameter will be always ignored.

The fix could be for row in result[start : start + end]:...

dsschult commented 7 years ago

Works for me, though we should test if this introduces slowness when on page 1000 :smile:

Not sure there is really a good workaround for high skip values, but maybe they won't happen very often and we can ignore it.

JadonKrys commented 7 years ago

I don't see a issue with high skip values since the lists are accessed by index. So, it isn't slower to access l[10000] than l[0], right?

dsschult commented 7 years ago

it's slower because the list doesn't exist until an item is asked for, so it actually has to go back and ask the DB for each item as it iterates/slices.

JadonKrys commented 7 years ago

Ah! Thanks, I didn't know that :)