search

billgraziano / xelogstash

Send SQL Server Extended Events to Logstash, Elastic Search, or JSON
Other
26 stars 11 forks source link

Trying to set it up, debug reports that it sends to logstash, but I don't see anything in elasticsearch? #90

Closed fribse closed 1 year ago

fribse commented 1 year ago

The logfile looks like this:

C:\sqlxewriter>sqlxewriter.exe -debug
INFO[0000] start: version: 1.7.6; git: v1.7.6; build: 2022-06-21T08:01:12-05:00
INFO[0000] log level: debug
DEBU[0000] app.program.start
DEBU[0000] desired config file: C:\sqlxewriter\sqlxewriter.toml
DEBU[0000] checking sources file: C:\sqlxewriter\sqlxewriter_sources.toml
INFO[0000] config file: C:\sqlxewriter\sqlxewriter.toml (2023-08-14T12:03:05+02:00)
INFO[0000] default poll interval: 60s
INFO[0000] sources: 1; default rows: 1
INFO[0000] sink: files: C:\sqlxewriter\events\sqlevents_YYYYMMDD_HH.events (keep: 4h0m0s)
INFO[0000] sink: logstashsink: host: logstash:5000
DEBU[0000] ls2: host: logstash:5000; addr: 10.100.1.48:5000
INFO[0000] localhost: polling interval: 60s              source=localhost
DEBU[0001] ALPHASQL session: system_health; events: 4  duration_ms=590 events=4 events_per_sec=6 instance=ALPHASQL session=system_health source=localhost
INFO[0001] metrics: alloc: 5.8mb; sys: 18.4mb; goroutines: 3; uptime: 0m1s; sources: 1
INFO[0001] stop signal received
DEBU[0001] sending cancel to pollers...
INFO[0001] all sinks closed
INFO[0001] application stopping

So for me it looks like it connects to the 'logstash' but I don't see anything in elasticsearch? Does the debug not send the data, but only does a connect? Or is something else wrong? The logstash and elasticsearch works, and I'm using it for a lot of other logs.

fribse commented 1 year ago

I just tried setting it up to use elasticsearch directly with SSL, username, and password, and that works, but I really wouldn't like to go directly to the elasticsearch, I have logstash adding monthstamps to the index names, so I can do lifecycle on the logs.

I looked up another server I have running with filebeat, connecting to logstash as well, and that is setup the same way, same url, port etc. Logstash port is unencrypted.

fribse commented 1 year ago

Found this in the log for logstash:

logstash              | [2023-08-14T10:03:14,128][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:65128] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-14T10:03:14,142][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:65128] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-14T10:03:14,147][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:65128] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102)
logstash              | [2023-08-14T10:03:14,147][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:65128] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105)
logstash              | [2023-08-14T10:03:14,147][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:65128] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 101 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 101)
logstash              | [2023-08-14T10:03:14,158][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:65128] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 108 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 108)
logstash              | [2023-08-14T10:06:13,251][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:65158] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-14T10:06:13,252][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:65158] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-14T10:06:13,265][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:65163] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-14T10:06:13,265][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:65163] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-14T10:06:13,267][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:65163] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102)
logstash              | [2023-08-14T10:06:13,310][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:65164] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-14T10:06:13,311][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:65164] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-14T10:09:47,471][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:65185] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-14T10:09:47,472][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:65185] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-14T10:09:47,473][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:65185] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102)
logstash              | [2023-08-14T10:09:47,473][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:65185] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105)
logstash              | [2023-08-14T10:09:47,473][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:65185] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 101 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 101)
logstash              | [2023-08-15T08:07:34,527][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:51738] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-15T08:07:34,528][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:51738] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-15T08:07:34,528][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:51738] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102)
logstash              | [2023-08-15T08:07:34,534][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:51738] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105)
logstash              | [2023-08-15T08:13:21,818][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:51800] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-15T08:13:21,820][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:51800] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-15T08:13:21,820][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:51800] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102)
logstash              | [2023-08-15T08:13:21,822][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:51800] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105)
logstash              | [2023-08-15T08:13:21,822][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:51800] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 101 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 101)
logstash              | [2023-08-15T08:39:02,449][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:52140] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-15T08:39:02,451][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:52140] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-15T08:39:02,475][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:52145] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-15T08:39:02,476][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:52145] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-15T08:39:02,477][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:52145] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102)
logstash              | [2023-08-15T08:39:02,502][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:52146] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-15T08:39:02,503][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:52146] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-15T08:39:02,504][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 172.18.0.7:5000, remote: 10.100.1.79:52146] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102)
logstash              | [2023-08-15T08:39:20,619][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:52148] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-15T08:39:20,619][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:52148] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-15T08:39:20,619][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:52148] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102)
logstash              | [2023-08-15T08:39:20,619][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:52148] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105)
logstash              | [2023-08-15T08:39:20,619][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:52148] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 101 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 101)
logstash              | [2023-08-15T08:39:20,619][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:52148] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 108 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 108)
logstash              | [2023-08-15T08:39:20,619][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:52148] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 100 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 100)
logstash              | [2023-08-15T08:48:15,531][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:52283] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-15T08:48:15,531][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:52283] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-15T08:48:15,531][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:52283] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102)
logstash              | [2023-08-15T08:48:15,531][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:52283] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105)
logstash              | [2023-08-15T08:48:15,531][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:52283] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 101 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 101)
logstash              | [2023-08-15T08:48:15,531][INFO ][org.logstash.beats.BeatsHandler][main][0ae7366b552aa72be240e80e886ea52cc484d39bbad92f9b7a97353823bae5b7] [local: 0.0.0.0:5000, remote: 10.100.1.79:52283] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 108 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 108)
logstash              | [2023-08-15T09:09:39,728][INFO ][org.logstash.beats.BeatsHandler][main][2e4fb0dd45b37681ccf2645f9529ab0bad290e4ed5e8cb2b5acfee12015badcc] [local: 172.18.0.7:5000, remote: 10.100.1.79:52537] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-15T09:09:39,736][INFO ][org.logstash.beats.BeatsHandler][main][2e4fb0dd45b37681ccf2645f9529ab0bad290e4ed5e8cb2b5acfee12015badcc] [local: 172.18.0.7:5000, remote: 10.100.1.79:52537] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-15T09:09:39,736][INFO ][org.logstash.beats.BeatsHandler][main][2e4fb0dd45b37681ccf2645f9529ab0bad290e4ed5e8cb2b5acfee12015badcc] [local: 172.18.0.7:5000, remote: 10.100.1.79:52537] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102)
logstash              | [2023-08-15T09:09:39,736][INFO ][org.logstash.beats.BeatsHandler][main][2e4fb0dd45b37681ccf2645f9529ab0bad290e4ed5e8cb2b5acfee12015badcc] [local: 172.18.0.7:5000, remote: 10.100.1.79:52537] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 105)
logstash              | [2023-08-15T09:09:39,737][INFO ][org.logstash.beats.BeatsHandler][main][2e4fb0dd45b37681ccf2645f9529ab0bad290e4ed5e8cb2b5acfee12015badcc] [local: 172.18.0.7:5000, remote: 10.100.1.79:52537] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 101 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 101)
logstash              | [2023-08-15T09:09:39,737][INFO ][org.logstash.beats.BeatsHandler][main][2e4fb0dd45b37681ccf2645f9529ab0bad290e4ed5e8cb2b5acfee12015badcc] [local: 172.18.0.7:5000, remote: 10.100.1.79:52537] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 108 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 108)
logstash              | [2023-08-15T09:09:39,737][INFO ][org.logstash.beats.BeatsHandler][main][2e4fb0dd45b37681ccf2645f9529ab0bad290e4ed5e8cb2b5acfee12015badcc] [local: 172.18.0.7:5000, remote: 10.100.1.79:52537] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 100 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 100)
logstash              | [2023-08-15T09:19:15,061][INFO ][org.logstash.beats.BeatsHandler][main][2e4fb0dd45b37681ccf2645f9529ab0bad290e4ed5e8cb2b5acfee12015badcc] [local: 172.18.0.7:5000, remote: 10.100.1.79:52626] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 123)
logstash              | [2023-08-15T09:19:15,062][INFO ][org.logstash.beats.BeatsHandler][main][2e4fb0dd45b37681ccf2645f9529ab0bad290e4ed5e8cb2b5acfee12015badcc] [local: 172.18.0.7:5000, remote: 10.100.1.79:52626] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 34)
logstash              | [2023-08-15T09:19:15,062][INFO ][org.logstash.beats.BeatsHandler][main][2e4fb0dd45b37681ccf2645f9529ab0bad290e4ed5e8cb2b5acfee12015badcc] [local: 172.18.0.7:5000, remote: 10.100.1.79:52626] Handling exception: io.netty.handler.codec.DecoderException: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102 (caused by: org.logstash.beats.InvalidFrameProtocolException: Invalid version of beats protocol: 102)

My logstash is 7.17.4, do I need to add an old logstash to handle this?

fribse commented 1 year ago

Ahh, it needs a tcp input, not a beats input, so I added this on a seperate port:

input {
        tcp {
                port => 5001
                type => "tcp-input"
        }
}

And now this part works

billgraziano commented 1 year ago

-debug just changes the logging level. Running it interactively just runs one loop and exits. You can install it as a service or run using the -loop parameter if in a container.