SoftInstigate / restheart

Rapid API Development with MongoDB
https://restheart.org
GNU Affero General Public License v3.0
805 stars 171 forks source link

Question > Shouldn't limit(x) be used instead of looping over cursor? #343

Closed Musikolo closed 5 years ago

Musikolo commented 5 years ago

Hi guys,

I reported a question some time ago, that end up with this enhancement. That was pretty good, but I still think it can be fine tuned even further.

Let's assume the following scenario:

So, according to the code we have here, the cursor will pull the first 100 documents, and code will return the first 25.

I think this can be improved if instead of counting the number items (_pagesize) in a loop, we would use .skip(toSkip).limit(pagesize), in other words:

            cursor = getFindIterable(coll, sortBy, filters, hint, keys);
            cursor.skip(toskip)
                  .limit(pagesize);

            MongoCursor<BsonDocument> mc = cursor.iterator();

            while (mc.hasNext()) {
                ret.add(mc.next());
            }

IMO, the advantage of this approach is that the cursor will just pull 25 items, instead of the full batch (100). Note that when documents have certain size, or for big collections, the improvement is noticeable.

Thoughts?

ujibang commented 5 years ago

Hi @Musikolo ,

thanks for the suggestion! This has been committed 7c8ee1e47bfc1e5d790993167d6a86f2607c5774

Note that it is currently in the branch next and will be in RESTHeart 4.0 since we are not going to update 3.x but for critical bugs.