search

chef / chef-server

Chef Infra Server is a hub for configuration data; storing cookbooks, node policies and metadata of managed nodes.
https://www.chef.io/chef/
Apache License 2.0
289 stars 209 forks source link

[terraform] external-elasticsearch scenario fails on ipv6 #1957

Open PrajaktaPurohit opened 4 years ago

PrajaktaPurohit commented 4 years ago 

null_resource.chef_server_config (remote-exec):   ================================================================================
--
  | null_resource.chef_server_config (remote-exec):   Recipe Compile Error in /var/opt/opscode/local-mode-cache/cookbooks/private-chef/recipes/default.rb
  | null_resource.chef_server_config (remote-exec):   ================================================================================
  |  
  | null_resource.chef_server_config (remote-exec):   RuntimeError
  | null_resource.chef_server_config (remote-exec):   ------------
  | null_resource.chef_server_config (remote-exec):   Failed to connect to elasticsearch service. Ensure node['private_chef']['opscode-solr4']['external_url'] is correct.
  | null_resource.chef_server_config (remote-exec):   Error connecting to http://elasticsearch.internal:9200 - Failed to open TCP connection to elasticsearch.internal:9200 (Connection timed out - connect(2) for "elasticsearch.internal" port 9200)
  |  
  | null_resource.chef_server_config (remote-exec):   Cookbook Trace:
  | null_resource.chef_server_config (remote-exec):   ---------------
  | null_resource.chef_server_config (remote-exec):     /var/opt/opscode/local-mode-cache/cookbooks/private-chef/libraries/helper.rb:91:in `rescue in elastic_search_major_version'
  | null_resource.chef_server_config (remote-exec):     /var/opt/opscode/local-mode-cache/cookbooks/private-chef/libraries/helper.rb:85:in `elastic_search_major_version'
  | null_resource.chef_server_config (remote-exec):     /var/opt/opscode/local-mode-cache/cookbooks/private-chef/libraries/helper.rb:118:in `es_index_definition'
  | null_resource.chef_server_config (remote-exec):     /var/opt/opscode/local-mode-cache/cookbooks/private-chef/recipes/opscode-solr4-external.rb:25:in `block in from_file'
  | null_resource.chef_server_config (remote-exec):     /var/opt/opscode/local-mode-cache/cookbooks/private-chef/recipes/opscode-solr4-external.rb:23:in `from_file'
  | null_resource.chef_server_config (remote-exec):     /var/opt/opscode/local-mode-cache/cookbooks/private-chef/recipes/default.rb:149:in `block in from_file'
  | null_resource.chef_server_config (remote-exec):     /var/opt/opscode/local-mode-cache/cookbooks/private-chef/recipes/default.rb:145:in `each'
  | null_resource.chef_server_config (remote-exec):     /var/opt/opscode/local-mode-cache/cookbooks/private-chef/recipes/default.rb:145:in `from_file'
  |  
  | null_resource.chef_server_config (remote-exec):   Relevant File Content:
  | null_resource.chef_server_config (remote-exec):   ----------------------
  | null_resource.chef_server_config (remote-exec):   /var/opt/opscode/local-mode-cache/cookbooks/private-chef/libraries/helper.rb:
  | null_resource.chef_server_config (remote-exec):
  | null_resource.chef_server_config (remote-exec):    84:      if node['private_chef']['opscode-solr4']['external'] && node['private_chef']['opscode-erchef']['search_provider'] == 'elasticsearch'
  | null_resource.chef_server_config (remote-exec):    85:        begin
  | null_resource.chef_server_config (remote-exec):    86:          client = Chef::HTTP.new(node['private_chef']['opscode-solr4']['external_url'])
  | null_resource.chef_server_config (remote-exec):    87:          response = client.get('')
  | null_resource.chef_server_config (remote-exec):    88:        rescue => e
  | null_resource.chef_server_config (remote-exec):    89:          # Perform a blind rescue because Net:HTTP throws a variety of exceptions - some of which are platform specific.
  | null_resource.chef_server_config (remote-exec):    90:          if current_request == max_requests
  | null_resource.chef_server_config (remote-exec):    91>>           raise "Failed to connect to elasticsearch service. Ensure node['private_chef']['opscode-solr4']['external_url'] is correct.\n#{e}"
  | null_resource.chef_server_config (remote-exec):    92:          else
  | null_resource.chef_server_config (remote-exec):    93:            # Chef HTTP logs the details in the debug log.
  | null_resource.chef_server_config (remote-exec):    94:            Chef::Log.error "Failed to connect to elasticsearch service #{current_request}/#{max_requests}. Retrying."
  | null_resource.chef_server_config (remote-exec):    95:            current_request += 1
  | null_resource.chef_server_config (remote-exec):    96:            sleep(current_request * 2) # Exponential back-off.
  | null_resource.chef_server_config (remote-exec):    97:            retry
  | null_resource.chef_server_config (remote-exec):    98:          end
  | null_resource.chef_server_config (remote-exec):    99:        end
  | null_resource.chef_server_config (remote-exec):   100:        begin
  | null_resource.chef_server_config (remote-exec):
  | null_resource.chef_server_config (remote-exec):   System Info:
  | null_resource.chef_server_config (remote-exec):   ------------
  | null_resource.chef_server_config (remote-exec):   chef_version=15.8.23
  | null_resource.chef_server_config (remote-exec):   platform=redhat
  | null_resource.chef_server_config (remote-exec):   platform_version=6.10
  | null_resource.chef_server_config (remote-exec):   ruby=ruby 2.6.5p114 (2019-10-01 revision 67812) [x86_64-linux]
  | null_resource.chef_server_config (remote-exec):   program_name=/opt/opscode/embedded/bin/chef-client
  | null_resource.chef_server_config (remote-exec):   executable=/opt/opscode/embedded/bin/chef-client
  |  
  | null_resource.chef_server_config (remote-exec):
  | null_resource.chef_server_config (remote-exec):   Running handlers:
  | null_resource.chef_server_config (remote-exec):   Running handlers complete
  | null_resource.chef_server_config (remote-exec):   Chef Infra Client failed. 1 resources updated in 34 minutes 07 seconds
  | null_resource.chef_server_config (remote-exec): [2020-04-07T20:41:57+00:00] FATAL: Stacktrace dumped to /var/opt/opscode/local-mode-cache/chef-stacktrace.out
  | null_resource.chef_server_config (remote-exec): [2020-04-07T20:41:57+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
  | null_resource.chef_server_config (remote-exec): [2020-04-07T20:41:57+00:00] FATAL: RuntimeError: Failed to connect to elasticsearch service. Ensure node['private_chef']['opscode-solr4']['external_url'] is correct.
  | null_resource.chef_server_config (remote-exec): Error connecting to http://elasticsearch.internal:9200 - Failed to open TCP connection to elasticsearch.internal:9200 (Connection timed out - connect(2) for "elasticsearch.internal" port 9200)
  |  
PrajaktaPurohit commented 4 years ago

The original issue was fixed, however posting to es for chef-server in ipv6 fails with an nxdomain error. The work done for this is tracked in https://github.com/chef/chef-server/pull/2024

PrajaktaPurohit commented 3 years ago

currently this scenario fails with a slightly different error:


null_resource.chef_server_config (remote-exec): echo -e '\nBEGIN ADD USER + ORGANIZATION\n'
--
  | null_resource.chef_server_config (remote-exec): + echo -e '\nBEGIN ADD USER + ORGANIZATION\n'
  |  
  | null_resource.chef_server_config (remote-exec): BEGIN ADD USER + ORGANIZATION
  |  
  |  
  | null_resource.chef_server_config (remote-exec): sudo chef-server-ctl user-create janedoe Jane Doe janed@example.com abc123 --filename "/home/${USER}/janedoe.pem"
  | null_resource.chef_server_config (remote-exec): + sudo chef-server-ctl user-create janedoe Jane Doe janed@example.com abc123 --filename /home/ubuntu/janedoe.pem
  | null_resource.chef_server_config: Still creating... [4m10s elapsed]
  | null_resource.chef_server_config (remote-exec): sudo chef-server-ctl org-create 4thcoffee 'Fourth Coffee, Inc.' --association_user janedoe --filename "/home/${USER}/4thcoffee-validator.pem"
  | null_resource.chef_server_config (remote-exec): + sudo chef-server-ctl org-create 4thcoffee 'Fourth Coffee, Inc.' --association_user janedoe --filename /home/ubuntu/4thcoffee-validator.pem
  | null_resource.chef_server_config (remote-exec): ERROR: Server returned error 500 for https://127.0.0.1/organizations, retrying 1/5 in 4s
  | null_resource.chef_server_config (remote-exec): ERROR: Conflict
  | null_resource.chef_server_config (remote-exec): Response: Organization already exists

Still needs investigation.

PrajaktaPurohit commented 3 years ago

Ensure that the scenario is added to the https://buildkite.com/chef/chef-umbrella-master-chef-server-full/builds/15 pipeline once fixed.