issue with beats-plugin 1.1..and graylog 1.3

[emb3dd3d]

On Wednesday, 10 February 2016 00:07:31 UTC+1, emb3dd3d wrote: I am not sure what to even look for to get this working. I configured the beats plugin in inputs with no SSL while testing.. and configured the filebeat collector to output logstash format.. It appears to be trying to input into Graylog-server but I get the above error.. there are no min and max settings for the beats input so I am at a loss.

2016-02-09T17:04:23.065-06:00 WARN [SimpleChannelHandler] EXCEPTION, please implement org.graylog.inputs.beats.server.EventHandler.exceptionCaught() for proper handling. java.lang.IndexOutOfBoundsException: Not enough readable bytes - Need 825110797, maximum is 86

[sivasamyk]

Please run filebeat tool with following switches and paste the output.

./filebeat -v -e -c <config file>

[sivasamyk]

@emb3dd3d any updates on this output of above command?

[emb3dd3d]

Sorry. ...it's been kinda hectic around here.. I promise to get to this todsy..thx fornthencheckup.

On Wed, Mar 2, 2016, 9:44 AM Sivasamy Kaliappan notifications@github.com wrote:

@emb3dd3d https://github.com/emb3dd3d any updates on this output of above command?

— Reply to this email directly or view it on GitHub https://github.com/sivasamyk/graylog-beats-plugin/issues/2#issuecomment-191294375 .

sent from Trio AXS 4G

[emb3dd3d]

Ok.. I finally got to the output you requested.. I didn't see a whole lot of information that looks to help you but maybe I missed something..

filebeat -v -e -c /etc/filebeat/filebeat.yml 2016/03/02 17:30:09.658200 geolite.go:24: INFO GeoIP disabled: No paths were set under output.geoip.paths 2016/03/02 17:30:13.931532 outputs.go:119: INFO Activated elasticsearch as output plugin. 2016/03/02 17:30:13.931599 file.go:39: INFO File output base filename set to: filebeat 2016/03/02 17:30:13.931631 file.go:50: INFO Rotate every bytes set to: 10485760 2016/03/02 17:30:13.931640 file.go:57: INFO Number of files set to: 7 2016/03/02 17:30:13.931988 outputs.go:119: INFO Activated file as output plugin. 2016/03/02 17:30:13.932201 publish.go:288: INFO Publisher name: itstage2.laserxxxx.com 2016/03/02 17:30:13.933157 async.go:78: INFO Flush Interval set to: 1s 2016/03/02 17:30:13.933195 async.go:84: INFO Max Bulk Size set to: 50 2016/03/02 17:30:13.933816 async.go:78: INFO Flush Interval set to: -1ms 2016/03/02 17:30:13.933837 async.go:84: INFO Max Bulk Size set to: -1 2016/03/02 17:30:13.933868 beat.go:147: INFO Init Beat: filebeat; Version: 1.1.0 2016/03/02 17:30:13.937597 beat.go:173: INFO filebeat sucessfully setup. Start running. 2016/03/02 17:30:13.937705 registrar.go:66: INFO Registry file set to: /var/lib/filebeat/registry 2016/03/02 17:30:13.937771 registrar.go:76: INFO Loading registrar data from /var/lib/filebeat/registry 2016/03/02 17:30:13.938562 prospector.go:127: INFO Set ignore_older duration to 24h0m0s 2016/03/02 17:30:13.938607 prospector.go:127: INFO Set scan_frequency duration to 10s 2016/03/02 17:30:13.938634 prospector.go:87: INFO Input type set to: log 2016/03/02 17:30:13.938644 prospector.go:127: INFO Set backoff duration to 1s 2016/03/02 17:30:13.938654 prospector.go:127: INFO Set max_backoff duration to 10s 2016/03/02 17:30:13.938676 prospector.go:107: INFO force_close_file is disabled 2016/03/02 17:30:13.938706 prospector.go:137: INFO Starting prospector of type: log 2016/03/02 17:30:13.938857 spooler.go:77: INFO Starting spooler: spool_size: 1024; idle_timeout: 5s 2016/03/02 17:30:13.939865 log.go:113: INFO Harvester started for file: /var/log/httpd/access_log 2016/03/02 17:30:13.940127 log.go:113: INFO Harvester started for file: /var/log/httpd/error_log 2016/03/02 17:30:13.941471 log.go:113: INFO Harvester started for file: /var/log/httpd/ssl_request_log 2016/03/02 17:30:13.942391 log.go:113: INFO Harvester started for file: /var/log/httpd/ssl_access_log 2016/03/02 17:30:13.942420 log.go:113: INFO Harvester started for file: /var/log/httpd/viper.ssl.access.log 2016/03/02 17:30:13.949418 log.go:113: INFO Harvester started for file: /db/9.3/data/pg_log/postgresql-2016-03-02_111119.log 2016/03/02 17:30:13.949467 log.go:113: INFO Harvester started for file: /db/9.3/data/pg_log/postgresql-2016-03-02_111228.log 2016/03/02 17:30:13.950237 crawler.go:78: INFO All prospectors initialised with 0 states to persist 2016/03/02 17:30:13.950379 log.go:113: INFO Harvester started for file: /db/9.3/data/pg_log/postgresql-2016-03-02_110158.log 2016/03/02 17:30:13.950853 registrar.go:83: INFO Starting Registrar 2016/03/02 17:30:13.950946 publish.go:88: INFO Start sending events to output 2016/03/02 17:30:13.951517 log.go:113: INFO Harvester started for file: /db/9.3/data/pg_log/postgresql-2016-03-02_111531.log 2016/03/02 17:30:16.558743 single.go:126: INFO Connecting error publishing events (retrying): Head http://192.xxx.xxx.xxx:5044: EOF 2016/03/02 17:30:16.558869 single.go:152: INFO send fail 2016/03/02 17:30:16.558889 single.go:159: INFO backoff retry: 1s 2016/03/02 17:30:19.254129 single.go:126: INFO Connecting error publishing events (retrying): Head http://192.xxx.xxx.xxx:5044: EOF 2016/03/02 17:30:19.254155 single.go:152: INFO send fail 2016/03/02 17:30:19.254167 single.go:159: INFO backoff retry: 2s 2016/03/02 17:30:24.434782 single.go:126: INFO Connecting error publishing events (retrying): Head http://192.xxx.xxx.xxx:5044: EOF 2016/03/02 17:30:24.434847 single.go:152: INFO send fail 2016/03/02 17:30:24.434859 single.go:159: INFO backoff retry: 4s

[sivasamyk]

From the above logs I see that you have activated elasticsearch as output plugin. Instead you need to enable logstash output in the filebeat.yml file and comment elastic search output.

Uncomment below lines in your /etc/filebeat/filebeat.yml

### Logstash as output
  logstash:
    # The Logstash hosts
    hosts: ["<Graylog server IP>:5044"]

and comment

### Elasticsearch as output
  #elasticsearch:
    # Array of hosts to connect to.
    # Scheme and port can be left out and will be set to the default (http and 9200)
    # In case you specify and additional path, the scheme is required: http://localhost:9200/path
    # IPv6 addresses should always be defined as: https://[2001:db8::1]:9200
   # hosts: ["localhost:9200"]

[emb3dd3d]

k coo.. weird.. it has that entry enabled : false right below.. but I see what you mean.. disabling and trying again..

Thanks..

[emb3dd3d]

will close unless I have any related issue.. thanks.. that enabled: false setting threw me off.

[sivasamyk]

Great. Happy logging :) On Mar 3, 2016 00:20, "emb3dd3d" notifications@github.com wrote:

Closed #2 https://github.com/sivasamyk/graylog-beats-plugin/issues/2.

— Reply to this email directly or view it on GitHub https://github.com/sivasamyk/graylog-beats-plugin/issues/2#event-575115059 .