search

lancachenet / logstash

A starter repo for logstash config files for lancache / monolithic
11 stars 16 forks source link

ELK 7.x support #9

Closed nightah closed 2 years ago

nightah commented 5 years ago

I tried upgrading my ELK stack to 7.x over the weekend without success.

I'm wondering if anybody has managed to get the configuration working with ELK 7.x? It looked to me like that my_index field for the indexes was not being created properly.

If anybody has had success deploying this on 7.x could you please share the appropriate logstash and filebeat config and an object export from Kibana?

nightah commented 4 years ago

Alright so I figured this out, didn't quite have to do with the my_index field, because of the changes between 6.x for logstash and filebeat the logstash.conf needed to change to accommodate the new and deprecated fields.

Let me know if you want me to submit a PR to make this 7.x compatible.

nightah commented 4 years ago

The following changes need to be made to make a 7.x compatible config:

https://github.com/lancachenet/logstash/blob/aea8a7c07da661062e2f779d6089c30237fdc277/logstash.conf#L9

Becomes: if [host][name] =~ "^squid" {

https://github.com/lancachenet/logstash/blob/aea8a7c07da661062e2f779d6089c30237fdc277/logstash.conf#L34

Becomes: if [host][name] == "lancache" {

https://github.com/lancachenet/logstash/blob/aea8a7c07da661062e2f779d6089c30237fdc277/logstash.conf#L36-L43 Needs to be removed.

https://github.com/lancachenet/logstash/blob/aea8a7c07da661062e2f779d6089c30237fdc277/logstash.conf#L51 Becomes: if [log][file][path] =~ "access" {

https://github.com/lancachenet/logstash/blob/aea8a7c07da661062e2f779d6089c30237fdc277/logstash.conf#L73 Becomes: } else if [log][file][path] =~ "error" {

VibroAxe commented 4 years ago

@nightah can you submit a PR, but create a new logstash7.conf and rename logstash to logstash6.conf. The changes you suggest don't feel like they are backwards compatible?

VibroAxe commented 4 years ago

@nightah can you submit a PR, but create a new logstash7.conf and rename logstash to logstash6.conf. The changes you suggest don't feel like they are backwards compatible?

nightah commented 4 years ago

They’re definitely not backwards compatible. There are breaking changes from 6.x to 7.x in filebeat and logstash.

I’ll submit a PR shortly.

nightah commented 4 years ago

@VibroAxe: I've got a question pending in that PR can you have a look please? My suggestion is that we rename it.

Secondly could someone export all objects related to lancache (including any squid objects)? I want to import what I'm missing and create a new export which is clean and clearly identifies all objects as relating to LanCache as part of that existing PR.