puppet newbie -- testing and not seeing config changes

[SteveDevOps]

I'm finally getting communications happening and able to invoke a manifest specifying elasticsearch install. However my config param aren't showing up in the default .yml. What am I missing? -- I'm testing on centos 7 vms, running PE 3.7.

My custom manifest:

node 'xxxx' {

    group { 'esadmin':
        ensure => 'present'
    }

    user { 'esadmin':
        ensure => 'present',
        groups => 'esadmin'
    }

    class {'elasticsearch':
        java_install => true,
        manage_repo => true,
        repo_version => '1.4',
        datadir => '/var/data/elasticsearch',
        elasticsearch_user => 'esadmin',
        elasticsearch_group => 'esadmin',
        config => {
            'node.name' => $::fqdn,
            'cluster' => {
                'name' =>   'esearch',
                'routing.allocation.awareness.attributes' => 'rack'
            },
            'index' => {
                'number_of_replicas' => '1',
                'number_of_shards' => '4'
            },
            'network' => {
                'host' => $::ipaddress
            }
        }

    }

}

[electrical]

Hi,

You will need to define an instance as specified in https://github.com/elasticsearch/puppet-elasticsearch#instances Hope it helps.

[SteveDevOps]

Thanks.. ok got that in place, and manifest changes pulled down / accepted by agent. Now I'm confused on how to access the "instance" via url. I have the default service off, I found the extra instance service on my centos vm, and I'm re-specifying http.port as 9200 under my config => for instance.

[root@pnode-test pnode-test]# cat elasticsearch.yml
### MANAGED BY PUPPET ###
---
cluster: 
  name: prod_esearch
  routing: 
    allocation: 
      awareness: 
        attributes: rack
discovery: 
  zen: 
    minimum_master_nodes: 1
    ping: 
      multicast: 
        enabled: false
      timeout: 30s
      unicast: 
        hosts: ["xxxxx"]
gateway: 
  expected_nodes: 1
  recover_after_nodes: 2
  recover_after_time: 5m
  type: local
http: 
  port: 9200
index: 
  number_of_replicas: 1
  number_of_shards: 5
network: 
  host: xxxxx
node: 
  master: true
  name: pnode-test
path: 
  data: /var/data/elasticsearch/pnode-test

[root@pnode-test pnode-test]# service elasticsearch-pnode-test start
Starting elasticsearch-pnode-test (via systemctl):         [  OK  ]

[electrical]

Hi,

You should be able to access it via the same IP and port as you would do with the default instance.

[SteveDevOps]

electrical -- I get nothing.. If I switch back to using default elasticsearch service it works but obviously doesn't have my puppet-specified config.

[root@pnode-test elasticsearch]# tree /etc/elasticsearch
/etc/elasticsearch
├── elasticsearch.yml
├── logging.yml
├── pnode-test
│   ├── elasticsearch.yml --- this one has my config for instance
│   └── logging.yml
└── templates_import

[root@pnode-test elasticsearch]# chkconfig --list

Note: This output shows SysV services only and does not include native
      systemd services. SysV configuration data might be overridden by native
      systemd configuration.

      If you want to list systemd services use 'systemctl list-unit-files'.
      To see services enabled on particular target use
      'systemctl list-dependencies [target]'.

elasticsearch-pnode-test    0:off   1:off   2:on    3:on    4:on    5:on    6:off
iprdump         0:off   1:off   2:on    3:on    4:on    5:on    6:off
iprinit         0:off   1:off   2:on    3:on    4:on    5:on    6:off
iprupdate       0:off   1:off   2:on    3:on    4:on    5:on    6:off
netconsole      0:off   1:off   2:off   3:off   4:off   5:off   6:off
network         0:off   1:off   2:on    3:on    4:on    5:on    6:off

[SteveDevOps]

my updated manifest from puppetmaster:

cat testesinstall.pp
node 'pnode-test.local' {

    group { 'esadmin':
        ensure => 'present'
    }

    user { 'esadmin':
        ensure => 'present',
        groups => 'esadmin'
    }

    file { '/var/data' :
        ensure => 'directory',
        owner => 'root',
        group => 'wheel',
        mode => 766
    }   

    file { '/var/data/elasticsearch' :
        ensure => 'directory',
        owner => 'esadmin',
        group => 'esadmin',
        mode => 766
    }

    class {'elasticsearch':
        java_install => true,
        manage_repo => true,
        repo_version => '1.4',
        datadir => '/var/data/elasticsearch',
        elasticsearch_user => 'esadmin',
        elasticsearch_group => 'esadmin'
    }

    elasticsearch::instance { 'pnode-test' :
        config => {
            'discovery.zen.minimum_master_nodes' => '1',
            'discovery.zen.ping.timeout' => '30s',
            'discovery.zen.ping.multicast.enabled' => 'false',
            'discovery.zen.ping.unicast.hosts' => '["xxxxx"]',
            'http.port' => '9200',
                        'cluster' => {
                                'name' =>   'prod_esearch',
                                'routing.allocation.awareness.attributes' => 'rack'
                        },
                        'index' => {
                                'number_of_replicas' => '1',
                                'number_of_shards' => '5'
                        },
                        'network' => {
                                'host' => $::ipaddress
                        },
            'node' => {
                'name' => 'pnode-test',
                'master' => 'true'
            },
            'gateway' => {
                'type' => 'local',
                'recover_after_nodes' => '2',
                'recover_after_time' => '5m',
                'expected_nodes' => '1'
            }
        }   

    }

    elasticsearch::plugin{'mobz/elasticsearch-head':
                        module_dir => 'head',
                        instances => ['pnode-test'],
        }

    elasticsearch::plugin{'elasticsearch/elasticsearch-analysis-kuromoji/2.4.1':
                        module_dir => 'analysis-kuromoji',
                        instances => ['pnode-test'],
        }

    elasticsearch::plugin{'elasticsearch/marvel/latest':
                        module_dir => 'marvel',
                        instances => ['pnode-test'],
        }

}

[electrical]

strange it didn't add the service to the list. i assumed it would.. did you see any errors with the puppet runs?

Must admit i haven't done any testing yet on Centos7.

[SteveDevOps]

hmm..the instance isn't logging anything new in /var/log/elasticsearch.log (only default instance is) -- do I also need log path that specified in instance config?

[electrical]

Logging should be added as well and automatically ( default its the cluster name )

[SteveDevOps]

weird.. I only have defaults:

[root@pnode-test elasticsearch]# pwd
/var/log/elasticsearch
[root@pnode-test elasticsearch]# ls -al
total 176
drwxr-xr-x.  2 elasticsearch elasticsearch   4096 Jan  6 09:17 .
drwxr-xr-x. 21 root          root            4096 Jan  5 15:21 ..
-rw-r--r--.  1 elasticsearch elasticsearch      0 Jan  5 16:16 elasticsearch_index_indexing_slowlog.log
-rw-r--r--.  1 elasticsearch elasticsearch      0 Jan  5 16:16 elasticsearch_index_search_slowlog.log
-rw-r--r--.  1 elasticsearch elasticsearch   5164 Jan  6 09:58 elasticsearch.log
-rw-r--r--.  1 elasticsearch elasticsearch 160967 Jan  5 16:19 elasticsearch.log.2015-01-05

so in a nutshell the module can't affect default instance config? only new instance? I only ask that because I am able to get default up and running.

[electrical]

That's correct. The default instance that comes with the package is not used at all. With the next release i will be removing the files so people don't get confused :-)

[SteveDevOps]

k. Is there anything else I'm missing in my ### MANAGED BY PUPPET ### .yml above?

[electrical]

file looks good. don't see anything wrong with it by the looks of it. It could be that by default we use the sysv style init script and not systemd one.

[SteveDevOps]

despite that, centos7 still shows it under old chkconfig --list and I was thinking would be compatible.. I've also tried re-specifying path.logs in instance config, no luck yet :( so close!

[SteveDevOps]

ahh found issue in messages and not sure why yet..

an  6 12:38:40 pnode-test elasticsearch-pnode-test: Starting elasticsearch-pnode-test: [  OK  ]
Jan  6 12:38:40 pnode-test systemd: Started LSB: This service manages the elasticsearch daemon.
Jan  6 12:38:40 pnode-test elasticsearch-pnode-test: {1.4.2}: pid Failed ...
Jan  6 12:38:40 pnode-test elasticsearch-pnode-test: - FileNotFoundException[/var/run/elasticsearch/elasticsearch-pnode-test.pid (Permission denied)]
Jan  6 12:38:40 pnode-test elasticsearch-pnode-test: java.io.FileNotFoundException: /var/run/elasticsearch/elasticsearch-pnode-test.pid (Permission denied)
Jan  6 12:38:40 pnode-test elasticsearch-pnode-test: at java.io.FileOutputStream.open(Native Method)
Jan  6 12:38:40 pnode-test elasticsearch-pnode-test: at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
Jan  6 12:38:40 pnode-test elasticsearch-pnode-test: at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
Jan  6 12:38:40 pnode-test elasticsearch-pnode-test: at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:156)
Jan  6 12:38:40 pnode-test elasticsearch-pnode-test: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)

[electrical]

Ah, that is my fault. With the current version i don't chown every directory related to ES yet. will be fixed in next release.

[SteveDevOps]

making progress, I fixed by getting rid of my

elasticsearch_user => 'esadmin',
elasticsearch_group => 'esadmin'

declaration.. so it uses default elasticsearch user for now.. this caused a chown -- back to elasticsearch user -- on everything on agent pull.

I can now call up new instance head plugin -- but it's blank I don't have any nodes showing.. hmm. cluster health: not connected.

[SteveDevOps]

houston we have liftoff on centos 7 ;) -- two other config issues.. I had pulled gateway settings (expected nodes up, etc) from our existing production config and got rid of that. Also I still had a custom data path set and permissioned for that other user I had made 'esadmin' nuked that as well -- woohoo.. we have a working head plugin ;)

[electrical]

Great to hear! i'll get the rights issue fixed as soon as possible. i hope to do a new release this month ( allot of changes and some bug fixes )

[electrical]

Hi,

I've just merged a fix into the master branch which should solve the rights issues.

Cheers.

[SteveDevOps]

Fantastic, thx electrical!