lusis / chef-logstash

Semi-official Logstash cookbook
Other
271 stars 353 forks source link

how to use config lwrp #426

Open dr3s opened 9 years ago

dr3s commented 9 years ago

hoping to get some help with the logstash config lwrp. can't seem to get the config lwrp to output any conf file.

logstash installs ok but the conf.d is empty my recipe https://gist.github.com/dr3s/c3652d8f144049c0ed6e my attributes https://gist.github.com/dr3s/1fc7d107107b0f7f07c2

dr3s commented 9 years ago
logstash-wolfe git:(develop) ✗ kitchen converge default-centos-64 --debug
-----> Starting Kitchen (v1.4.2)
-----> Converging <default-centos-64>...
       Preparing files for transfer
       Preparing dna.json
       Resolving cookbook dependencies with Berkshelf 3.3.0...
       Removing non-cookbook files before transfer
       Preparing solo.rb
-----> Chef Omnibus installation detected (install only if missing)
       Transferring files to <default-centos-64>
       Starting Chef Client, version 12.4.1
       Compiling Cookbooks...
       Converging 4 resources
       Recipe: logstash-wolfe::default
       [2015-08-27T14:55:16+00:00] WARN: Using an LWRP provider by its name (Ark) directly is no longer supported in Chef 12 and will be removed.  Use Chef::ProviderResolver.new(node, resource, action) instead.
        (up to date)
         * user[logstash] action create (up to date)
         * group[logstash] action create (up to date)
       Recipe: ark::default
        (up to date)
         * yum_package[autoconf] action install (up to date)
        (up to date)
        (up to date)
         * yum_package[make] action install (up to date)
        (up to date)

           * directory[/opt/logstash/server] action create (up to date)
        (up to date)
           * execute[unpack /tmp/kitchen/cache/server.tar.gz] action nothing (skipped due to action :nothing)
           * execute[set owner on /opt/logstash/server] action nothing (skipped due to action :nothing)
            (up to date)
         * directory[/opt/logstash/server/bin] action create (up to date)
         * directory[/opt/logstash/server/etc] action create (up to date)
         * directory[/opt/logstash/server/lib] action create (up to date)
         * directory[/opt/logstash/server/log] action create (up to date)
         * directory[/opt/logstash/server/tmp] action create (up to date)
         * directory[/opt/logstash/server/etc/conf.d] action create (up to date)
         * directory[/opt/logstash/server/patterns] action create (up to date)
       Recipe: logrotate::default
        (up to date)
        (up to date)
       Recipe: <Dynamically Defined Resource>
         * template[/etc/logrotate.d/logstash_server] action create (up to date)
       Recipe: logstash-wolfe::default
        (up to date)
       Recipe: runit::default
         * service[runit] action nothing (skipped due to action :nothing)
         * execute[start-runsvdir] action nothing (skipped due to action :nothing)
         * execute[runit-hup-init] action nothing (skipped due to action :nothing)

           * yum_package[pygpgme] action install (up to date)
           * log[pygpgme_warning] action write (skipped due to not_if)
        (skipped due to not_if)
        (up to date)
        (up to date)
           * execute[yum-makecache-imeyer_runit_] action nothing (skipped due to action :nothing)
           * ruby_block[yum-cache-reload-imeyer_runit_] action nothing (skipped due to action :nothing)
            (up to date)
         * yum_package[runit] action install (up to date)
       Recipe: <Dynamically Defined Resource>
         * service[logstash_server] action nothing (skipped due to action :nothing)

        (up to date)
        (up to date)
        (up to date)
           * directory[/etc/sv/logstash_server/log/main] action create (up to date)
        (up to date)
        (up to date)
        (up to date)
           * ruby_block[zap extra env files] action run (skipped due to only_if)
           * directory[/etc/sv/logstash_server/control] action create (up to date)
        (up to date)
           * file[/etc/sv/logstash_server/down] action nothing (skipped due to action :nothing)
           * link[/etc/service/logstash_server] action create (up to date)
            (up to date)
       Recipe: logstash-wolfe::default
        (up to date)
         * logstash_pattern[server] action create (up to date)

       Running handlers:
       Running handlers complete
       Chef Client finished, 0/41 resources updated in 27.854049045 seconds
       Finished converging <default-centos-64> (0m41.18s).
a-zen commented 8 years ago

I'm having the same problem. Although it says config is up to date, there is none created.

spuder commented 8 years ago

Github issues isn't ment for getting support.

You will find lots of helpful people over on the official logstash discussion boards.

https://discuss.elastic.co/

jmccann commented 8 years ago

https://github.com/rackspace-cookbooks/elkstack/blob/master/recipes/logstash.rb#L45-L51 really helped me to start and understand how logstash_config works since it's not documented in this repo's README.md.

Basically, logstash_config if empty does nothing. You need to provide the templates and template_variables that it will then consume to deploy to the correct log stash instance.

I hope this helps you as I struggled personally for 1-2 hours on it. 😭

Example:

logstash_config 'server' do
  templates_cookbook 'elasticstack_internal'
  templates 'output_elasticsearch' => 'output_elasticsearch.conf.erb',
            'filename' => 'template_name.erb'
  variables elasticsearch_ip: '127.0.0.1', another_variable: 'some value'
  notifies :restart, 'logstash_service[server]'
end

On a side note in response to @spuder comment, I've seen github issues used for support and think it's up to the discretion of the maintainer to decide if they want to allow that or not. There is no global rule AFAIK.

dvinograd commented 8 years ago

I think this should've been documented from the start, to save endless hours and headache of anyone trying to use this cookbook. There's no documentation or example code to go from. Thanks @jmccann for providing the only working answer.