search

robcowart / elastiflow

Network flow analytics (Netflow, sFlow and IPFIX) with the Elastic Stack
Other
2.48k stars 592 forks source link

Error when starting Logstash 7.0 with elastiflow 3.4.1 #289

Closed skaldenhoven closed 5 years ago

skaldenhoven commented 5 years ago

Probably already known that the config doesn't work with Logstash 7.0 (yet). I know it is not supported yet for 7.0 but just performed the upgrade of the stack and wanted to mention the issue.

When logstash starts parsing the configuration files it terminates with the following error. Invalid FieldReference:[flow][server_city]]`

As per https://www.elastic.co/guide/en/elastic-stack/current/logstash-breaking-changes.html Field Reference Parser is more strict so I guess an adjustment needs to take place.

pobradovic08 commented 5 years ago

Have the same issue. Also, installing elastiflow.template.json template to elasticsearch fails 

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "Root mapping definition has unsupported parameters: [_default_ : {...}]"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [_default_ : {...}]",
    "caused_by": {
      "type": "mapper_parsing_exception",
      "reason": "Root mapping definition has unsupported parameters:  [_default_ : {...}]"
    }
  },
  "status":400
}
Chernomazov commented 5 years ago

293 - Here are patched elastiflow.template.json and elastiflow.kibana.7.0.x.json to support ELK 7.0.

skaldenhoven commented 5 years ago

This is also fixed with #297 Just made the change and 2 additional ones which also had double brackets at the end and then the pipeline would start up fine.

geudrik commented 5 years ago

@skaldenhoven if you've pushed the typo fix in the logstash yaml with additional changes, I can just close my PR. No sense doubling up. The one change I made was the only that caused my configset to break (I only use IPFIX, so I've disabled a handful of the configs)

geudrik commented 5 years ago

Oh, maybe I've read that wrong. Can you point out where else you saw double brackets? I can add it in to mine if you're not submitting your own PR

skaldenhoven commented 5 years ago

On lines 739 and 796 client_city]]

geudrik commented 5 years ago

Thanks! PR updated :)

skaldenhoven commented 5 years ago

Super!