rundeck 2.6

[MelanieGault]

Hi,

Is this plugin supposed to work with rundeck 2.6.4 ? Thanks !

Mélanie

[gschueler]

yes, are you having an issue with it?

[MelanieGault]

Hi,

Yes I can't make it work with rundeck 2.6.4 on centos 6.7.

In rundeck-config.properties I wrote (I also try in the groovy file) : rundeck.execution.logs.streamingWriterPlugins=LogstashPlugin

I copy the plugin groovy file in the appropriate directory

In project.properties I wrote project.plugin.StreamingLogWriter.LogstashPlugin.port=9700 project.plugin.StreamingLogWriter.LogstashPlugin.host=localhost

I have the following in my logstash config (it works for all syslog files)

input { 
  file {
    type => "syslog"
    path => [ "/var/log/secure",  "/var/log/maillog", "/var/log/cron", "/var/log/messages", "/var/log/syslog"]
  }

  tcp {
    debug => true
    format => "json"
    mode => server
    port => 9700
    tags => ["rundeck"]
    type => "rundeck"
  }
 } 
output {
              redis { host =>  ["redissrv1:6379","redissrv2:6379"]
              data_type => "list" 
              key => "logstash"
            }
}

with netstat, I see that logstash is really listening on port 9700, but I don't have any logs for rundeck jobs run in my indexes.

I don't have any error log in rundeck logfiles, I can't see anything with a tcpdump on port 9700.

That's why I wonder if the plugin is ok with this version.

Do you have any clues to make it work ? Thanks ! Mélanie

[MelanieGault]

Hi,

I am on rundeck 2.6.6, on centos 6.7, my /etc/rundeck/rundeck-config.groovy

...
rundeck.execution.logs.streamingWriterPlugins="LogstashPlugin"

my /etc/rundeck/framework.properties

...
framework.plugin.StreamingLogWriter.LogstashPlugin.host = localhost
framework.plugin.StreamingLogWriter.LogstashPlugin.port = 9700

I had nothing in audit.log but I disabled selinux to be 100% sure to have the simpliest conf...

Rundeck in debug mode : no log seen to explain comportment. I double or triple checked logstash conf and try a telent : string send to 9700 port are shiped well to elasticsearch.

I don't know groovy, but I tried to add log to confirm that the logstash connexion is really opened. The socket is open but... the event can't be send:

ERROR DisablingLogWriter: Failed addEvent for StreamingLogWriter(LogstashPlugin), disabling. Cannot get property 'count' on null object

So, I understand that context object is null, but I don't understand what is wrong in my conf that could have this consequence.

Thanks for your help. Mélanie.

[gschueler]

Hi, sorry it is not working for you.

Can you try enabling log4j DEBUG level for the "rundeck.services.logging" logger? this might help narrow down the issue. (in log4j.properties append log4j.logger.rundeck.services.logging=DEBUG)

I will try to reproduce the issue

[gschueler]

I was able to setup up the plugin and see it work in Logstash 2.3.2:

rundeck-config.properties:

...
rundeck.execution.logs.streamingWriterPlugins=LogstashPlugin

framework.properties:

...
framework.plugin.StreamingLogWriter.LogstashPlugin.port=9700
framework.plugin.StreamingLogWriter.LogstashPlugin.host=localhost

rundeck-logstash.conf:

input {

  tcp {  
    codec => "json"
    host => "localhost"
    mode => server
    port => 9700
    tags => ["rundeck"]
    type => "rundeck"
  }

}

output { 
  stdout { }
}

running logstash and example Rundeck execution output to logstash:

$ logstash-2.3.2/bin/logstash -f ../config/rundeck-logstash.conf 
Settings: Default pipeline workers: 8
Pipeline main started
2016-04-26T16:41:08.829Z 127.0.0.1 %{message}
2016-04-26T16:41:08.830Z 127.0.0.1 %{message}
2016-04-26T16:41:08.831Z 127.0.0.1 hi this is node madmartigan.local
2016-04-26T16:41:09.070Z 127.0.0.1 %{message}
2016-04-26T16:41:09.072Z 127.0.0.1 %{message}
2016-04-26T16:41:09.076Z 127.0.0.1 Execution 169 finished.

Note: the example logstash config provided is not compatible with logstash 2.3.2, it was written for earlier version. I had to run with the -t flag $ logstash-2.3.2/bin/logstash -f ../config/rundeck-logstash.conf -t to validate the config first and fix the errors.