search

mongoid / moped

A MongoDB driver for Ruby
https://mongoid.github.io/old/en/moped/
MIT License
201 stars 153 forks source link

Moped errors out with Cursor not found on a long running DB task #233

Open sriranggd opened 10 years ago

sriranggd commented 10 years ago

I was running a long running DB migration task and moped exited with the following error and stack

        The operation: "GET MORE"
        failed with error "cursor 5014917976516347150 not found"
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/moped-1.4.3/lib/moped/node.rb:195:in `get_more'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/moped-1.4.3/lib/moped/cursor.rb:44:in `get_more'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/moped-1.4.3/lib/moped/cursor.rb:29:in `each'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/moped-1.4.3/lib/moped/query.rb:76:in `each'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/moped-1.4.3/lib/moped/query.rb:76:in `each'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/bundler/gems/mongoid-e10f5ca2ceba/lib/mongoid/contextual/mongo.rb:118:in `block in each' 
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/bundler/gems/mongoid-e10f5ca2ceba/lib/mongoid/contextual/mongo.rb:615:in `selecting'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/bundler/gems/mongoid-e10f5ca2ceba/lib/mongoid/contextual/mongo.rb:117:in `each'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/bundler/gems/mongoid-e10f5ca2ceba/lib/mongoid/contextual.rb:19:in `each'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/bundler/gems/mongoid-e10f5ca2ceba/lib/mongoid/findable.rb:15:in `each'
        /home/ubuntu/www/apps/shopping-addon/releases/20131118183338/db/migrate/20131108145130_add_counts_to_user.rb:5:in `up'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/mongoid_rails_migrations-1.0.1/lib/mongoid_rails_migrations/active_record_ext/migrations.rb:85:in `block in migrate'        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/mongoid_rails_migrations-1.0.1/lib/mongoid_rails_migrations/active_record_ext/migrations.rb:85:in `migrate'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/mongoid_rails_migrations-1.0.1/lib/mongoid_rails_migrations/active_record_ext/migrations.rb:169:in `migrate'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/mongoid_rails_migrations-1.0.1/lib/mongoid_rails_migra
        tions/active_record_ext/migrations.rb:319:in `block in migrate'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/mongoid_rails_migrations-1.0.1/lib/mongoid_rails_migrations/active_record_ext/migrations.rb:297:in `each'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/mongoid_rails_migrations-1.0.1/lib/mongoid_rails_migrations/active_record_ext/migrations.rb:297:in `migrate'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/mongoid_rails_migrations-1.0.1/lib/mongoid_rails_migrations/active_record_ext/migrations.rb:203:in `up'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/mongoid_rails_migrations-1.0.1/lib/mongoid_rails_migrations/active_record_ext/migrations.rb:188:in `migrate'
        /home/ubuntu/www/apps/shopping-addon/shared/bundle/ruby/2.0.0/gems/mongoid_rails_migrations-1.0.1/lib/mongoid_rails_migrations/mongoid_ext/railties/database.rake:42:in `block (2 levels) in <top (required)>'        Tasks: TOP => db:migrate
        (See full trace by running task with --trace)

The mongoid version is a non-release version pointing to a particular commit. Moped version is 1.4.3.

This error occured after about 95200 documents were fetched and processed.

I am connecting to a MongoDB replica set with one primary and one secondary, and I haven't made modifications to mongoid.yml about reading from secondary. So I believe reading from secondary is set to true.

I haven't yet got a chance to reproduce this purely with Moped (i.e. without using Mongoid). I will try and do that and post the results here.

Ruby is 2.0.0p247. This was run from a EC2 micro instance running Ubuntu 12.04.3 LTS Server.

MongoDB being used is from Mongolab.com, running their dedicated node plan.

Let me know if any more information is needed.

dmitrypol commented 9 years ago

I usually solve these issues by putting no_timeout in my query