sous-chefs / sc-mongodb

Development repository for the sc-mongodb cookbook
https://supermarket.chef.io/cookbooks/sc-mongodb
Apache License 2.0
75 stars 91 forks source link

single node RS search fails with http 400 #167

Closed st-h closed 6 years ago

st-h commented 7 years ago

when there only is a single node in the replicaset (for instance when running a node with mms monitoring and backup enabled), the recipe fails with the autoconfigure setting node['mongodb']['auto_configure']['replicaset'] = true, returning http 400 "Bad Request"

when autoconfigure is set to false, the recipe runs successfully, however the replicaset is not initialized.

[2017-06-30T18:05:00+02:00] INFO: HTTP Request Returned 400 Bad Request: error

  ================================================================================
  Recipe Compile Error in /var/chef/cache/cookbooks/myapp/recipes/default.rb
  ================================================================================

  Net::HTTPServerException
  ------------------------
  400 "Bad Request"

  Cookbook Trace:
  ---------------
    /var/chef/cache/cookbooks/sc-mongodb/definitions/mongodb.rb:236:in `block in from_file'
    /var/chef/cache/cookbooks/sc-mongodb/recipes/replicaset.rb:36:in `from_file'
    /var/chef/cache/cookbooks/myapp/recipes/default.rb:163:in `from_file'

  Relevant File Content:
  ----------------------
  /var/chef/cache/cookbooks/sc-mongodb/definitions/mongodb.rb:

  229:      notifies :create, 'ruby_block[config_sharding]', :immediately if new_resource.is_mongos && new_resource.auto_configure_sharding
  230:      # we don't care about a running mongodb service in these cases, all we need is stopping it
  231:      ignore_failure true if new_resource.name == 'mongodb'
  232:    end
  233:  
  234:    # replicaset
  235:    if new_resource.is_replicaset && new_resource.auto_configure_replicaset
  236>>     rs_nodes = search(
  237:        :node,
  238:        "mongodb_cluster_name:#{new_resource.cluster_name} AND "\
  239:        'mongodb_is_replicaset:true AND '\
  240:        "mongodb_config_mongod_replication_replSetName:#{new_resource.replicaset_name} AND "\
  241:        "chef_environment:#{node.chef_environment}"
  242:      )
  243:  
  244:      ruby_block 'config_replicaset' do
  245:        block do

  System Info:
  ------------
  chef_version=12.21.3
  platform=ubuntu
  platform_version=16.04
  ruby=ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
  program_name=chef-client worker: ppid=25317;start=18:04:54;
  executable=/opt/chef/bin/chef-client

  Running handlers:
[2017-06-30T18:05:00+02:00] ERROR: Running exception handlers
  Running handlers complete
[2017-06-30T18:05:00+02:00] ERROR: Exception handlers complete
  Chef Client failed. 0 resources updated in 06 seconds
[2017-06-30T18:05:00+02:00] INFO: Sending resource update report (run-id: ff7089f9-fde6-49de-ae93-59f3008544a2)
[2017-06-30T18:05:01+02:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2017-06-30T18:05:01+02:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2017-06-30T18:05:01+02:00] ERROR: 400 "Bad Request"

chef-stacktrace.out:

Generated at 2017-06-30 18:05:01 +0200
Net::HTTPServerException: 400 "Bad Request"
/opt/chef/embedded/lib/ruby/2.3.0/net/http/response.rb:120:in `error!'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/http.rb:150:in `request'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/http.rb:115:in `get'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/search/query.rb:161:in `call_rest_service'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/search/query.rb:88:in `search'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/dsl/data_query.rb:42:in `search'
/var/chef/cache/cookbooks/sc-mongodb/definitions/mongodb.rb:236:in `block in from_file'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/dsl/definitions.rb:39:in `instance_eval'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/dsl/definitions.rb:39:in `evaluate_resource_definition'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/dsl/definitions.rb:14:in `mongodb_instance'
/var/chef/cache/cookbooks/sc-mongodb/recipes/replicaset.rb:36:in `from_file'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/mixin/from_file.rb:30:in `instance_eval'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/mixin/from_file.rb:30:in `from_file'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/cookbook_version.rb:233:in `load_recipe'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/run_context.rb:338:in `load_recipe'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/run_context.rb:294:in `block in include_recipe'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/run_context.rb:293:in `each'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/run_context.rb:293:in `include_recipe'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/dsl/include_recipe.rb:26:in `include_recipe'
/var/chef/cache/cookbooks/myapp/recipes/default.rb:163:in `from_file'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/mixin/from_file.rb:30:in `instance_eval'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/mixin/from_file.rb:30:in `from_file'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/cookbook_version.rb:233:in `load_recipe'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/run_context.rb:338:in `load_recipe'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/run_context/cookbook_compiler.rb:141:in `block in compile_recipes'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/run_context/cookbook_compiler.rb:138:in `each'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/run_context/cookbook_compiler.rb:138:in `compile_recipes'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/run_context/cookbook_compiler.rb:75:in `compile'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/run_context.rb:187:in `load'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/policy_builder/expand_node_object.rb:97:in `setup_run_context'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/client.rb:513:in `setup_run_context'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/client.rb:281:in `run'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/application.rb:295:in `block in fork_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/application.rb:283:in `fork'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/application.rb:283:in `fork_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/application.rb:248:in `block in run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/local_mode.rb:44:in `with_server_connectivity'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/application.rb:236:in `run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/application/client.rb:464:in `sleep_then_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/application/client.rb:451:in `block in interval_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/application/client.rb:450:in `loop'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/application/client.rb:450:in `interval_run_chef_client'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/application/client.rb:434:in `run_application'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/lib/chef/application.rb:59:in `run'
/opt/chef/embedded/lib/ruby/gems/2.3.0/gems/chef-12.21.3/bin/chef-client:26:in `<top (required)>'
/usr/bin/chef-client:57:in `load'
/usr/bin/chef-client:57:in `<main>'
ccrebolder commented 7 years ago

I just ran into this. Got it to compile by setting node.default['mongodb']['config']['mongod']['replication']['replSetName'] = 'something' (still getting errors but seems a different issue now)

JustLeo commented 7 years ago

Did you resolve it? I'm facing the same issue.

ccrebolder commented 7 years ago

No, I gave up and wrote an internal cookbook for mongodb that is simpler since it only has to fulfill our use case (replicaset with auth, but no sharding or MMS). Sorry I can't be of more help.

st-h commented 7 years ago

@JustLeo I could not find any time to look further into it so far and unfortunately it seems it won't get much better in the near future. I think there are at least two options:

We are going with option two for now, but this isn't quite production code yet, so we are ok with that until we can find the time or this gets more urgent for us.

DrHappyAngry commented 6 years ago

I don't think this is a bug but the documentation could be bit more clear. If you're setting the node.default['mongodb']['config']['mongod']['replication']['replSetName'] attribute in a wrapper cookbook, like I was trying to do, it'll bomb out at the beginning of the chef run because the attribute won't get set until the recipe in the wrapper runs. Setting the attribute in the role or environment, so the attribute is set at the start of the chef run will get you past the 400 error. Now if I can only figure out why enabling replication breaks adding users, I'd be in good shape.

st-h commented 6 years ago

@DrHappyAngry sorry about the late reply, but I can not really reproduce this. I have all the attributes set in an attributes file. However when I converge with up to date chef, I receive a different error:

 ruby_block[config_replicaset] action run

           ================================================================================
           Error executing action `run` on resource 'ruby_block[config_replicaset]'
           ================================================================================

           NoMethodError
           -------------
           undefined method `[]' for nil:NilClass

           Cookbook Trace:
           ---------------
           /tmp/kitchen/cache/cookbooks/sc-mongodb/libraries/mongodb.rb:74:in `configure_replicaset'
           /tmp/kitchen/cache/cookbooks/sc-mongodb/definitions/mongodb.rb:246:in `block (3 levels) in from_file'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cache/cookbooks/sc-mongodb/definitions/mongodb.rb

           244:     ruby_block 'config_replicaset' do
           245:       block do
           246:         MongoDB.configure_replicaset(node, replicaset_name, rs_nodes) unless new_resource.replicaset.nil?
           247:       end
           248:       action :nothing
           249:     end
           250: 

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cache/cookbooks/sc-mongodb/definitions/mongodb.rb:244:in `block in from_file'

           ruby_block("config_replicaset") do
             params {:mongodb_type=>"mongod", :action=>[:enable, :start], :logpath=>"/var/log/mongodb/mongod.log", :configservers=>[], :replicaset=>true, :notifies=>[], :not_if=>[], :name=>"mongod"}
             action [:nothing]
             default_guard_interpreter :default
             declared_type :ruby_block
             cookbook_name "sc-mongodb"
             recipe_name "replicaset"
             block #<Proc:0x0000000004eff6d8@/tmp/kitchen/cache/cookbooks/sc-mongodb/definitions/mongodb.rb:245>
             block_name "config_replicaset"
           end

           System Info:
           ------------
           chef_version=14.2.0
           platform=ubuntu
           platform_version=16.04
           ruby=ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
           program_name=/opt/chef/bin/chef-client
           executable=/opt/chef/bin/chef-client

maybe I was missing a param back then, as other issues like #178 seem to indicate. However, the good news: the above mentioned error seems to be fixed, but not yet released as mentioned here #197 and here #186.

lock[bot] commented 5 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.