batch_size is ignored

[amw]

It seems batch_size option is ignored in mongoid/origin.

i = 0; Sample.all.batch_size(2).each {|s| i += 1; Rails.logger.info(i); break if i > 1100}

Looking at rails log I would expect seeing GET MORE command every two documents, instead I see only one after 101 document (MongoDB default behavior). Like this:

  MOPED: 127.0.0.1:27017 QUERY        database=amw_development collection=samples selector={} flags=[] limit=0 skip=0 fields=nil (48.2831ms)
1
2
...
101
  MOPED: 127.0.0.1:27017 GET_MORE     database=amw_development collection=samples limit=0 cursor_id=1976664482462204574 (1515.9869ms)
102
...
1100
1101

Note that I set the batch size to 2 only for the purpose of this example. In my app I tried it with a value of 100.

This is a pretty big issue for me as my sample processing times out when I crunch through 1000 samples without touching the cursor. I don't want to set the no-timeout option on the cursor to avoid risking memory leaks if my app dies.

[sadowski]

I'm also seeing this issue.

[amw]

@durran Have you seen this?

[durran]

@amw Yes I am looking into it. It's actually not an Origin issue - it's that Moped doesn't handle this and Mongoid isn't passing the option through so I've got to fix it in both those repos.

[gshiftlabs-dbuttineau]

Looks like no_timeout isn't being passed either, at least I'm still seeing a number of cursor timeouts when specifying no_timeout on the query.

[gshiftlabs-dbuttineau]

Submitted a pull request to Moped and Mongoid that roughly implements this. Likely something I missed but it passes spec and seems to function properly.

Moped: https://github.com/mongoid/moped/pull/125 Mongoid: https://github.com/mongoid/mongoid/pull/2610

[durran]

This has been handled in Mongoid and Moped.