sous-chefs / elasticsearch

Development repository for the elasticsearch cookbook
https://supermarket.chef.io/cookbooks/elasticsearch
Other
881 stars 599 forks source link

cluster.initial_master_nodes #746

Open craffop opened 4 years ago

craffop commented 4 years ago

Just updated our stack to 7.x and after going through the deprecation I think I have a cluster all setup. However after a little back and forth with support it seems the cluster setup has changed quite a bit. https://www.elastic.co/blog/a-new-era-for-cluster-coordination-in-elasticsearch I figured this out and already have cluster.initial_master_nodes set via configuration item. However I was told this should only be set for the initial deployment of a cluster and that you should take this out after that. I have poked around but not sure if I have missed something in what the best way is to initially bootstrap a cluster then have this setting removed....

ernestas-poskus commented 4 years ago
  {
    discovery: {
      seed_hosts: es7_unicast_masters
    },
    cluster: {
      # Cluster needs at least 1 initial master when bootstrapping for
      # the first time
      initial_master_nodes: es7_unicast_masters(join: false).first
    }
  }

you can leave the config, in cases of complete cluster shutdown you will need it again. The trick here is to discover master using chef search(:) API, then sort the hosts and select first one for initial_master_nodes. Thats it.

craffop commented 4 years ago

Thank you

  {
    discovery: {
      seed_hosts: es7_unicast_masters
    },
    cluster: {
      # Cluster needs at least 1 initial master when bootstrapping for
      # the first time
      initial_master_nodes: es7_unicast_masters(join: false).first
    }
  }

you can leave the config, in cases of complete cluster shutdown you will need it again. The trick here is to discover master using chef search(:) API, then sort the hosts and select first one for initial_master_nodes. Thats it.

Thank you,

However, I think I unserstand what you are getting at, I just seem to have a bit of a gap in understanding as to what you are doing. I have worked with some basic chef search, but mainly through knife.

Here is a snippet of my recipe to deploy the config: configuration( 'action.auto_create_index' => '.security,.security-6,.watches,.triggered_watches,.watcher-history-*,.monitoring-*', 'action.destructive_requires_name' => true, 'cluster.name' => node['sc-smartsearch']['clustername'], 'cluster.initial_master_nodes' => node['sc-smartsearch']['hosts'].split(','), 'discovery.seed_hosts' => node['sc-smartsearch']['hosts'].split(','),

I tried to narrow this down by doing some node searches but did not get anything back. so not trying to get free chef help or anything but would appreciate a bit more information on how to do this :) Can not tell if you added this to a role or running this in the recipe.

ernestas-poskus commented 4 years ago

I tried to narrow this down by doing some node searches but did not get anything back.

use chef search API to find all masters, map found masters into array and sort them ["master1:9200", "master2:9200", "master3:9200"]

assign above array to seed_hosts and set initial_master_nodes to first element from array of masters.