Issue with iterative IF statements

[nameraj]

I have issue with iterative IF statements. I want to use different FIX.XML for different messages same as example provided. but I am getting below errors. Logstash is crashing with this errors. Everything works perfect when I only use one FIX.XML and take out if statements. would any one please help?

Errors :

[2017-10-20T15:15:29,635][INFO ][logstash.filters.fixprotocol] Using version 0.1.x filter plugin 'fix_protocol'. This plugin isn't well supported by the community and likely has no maintainer. [2017-10-20T15:15:29,851][ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Constructor invocation failed: does not have a 'required' attribute"}

My Logstash config: ...... if "FIX" in [message] { grok { patterns_dir => [ "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-patterns-core-5.3.0/patterns" ] match => ["message","%{TIMESTAMP_ISOFIX:timestamp} %{GREEDYDATA:fix_session} - >?<? %{GREEDYDATA:fix_string}"] }

if "vendor1" in [message] { fix_protocol { fix_message => fix_string data_dictionary_path => "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-fix_protocol-0.3.2/FIX.xml" } if "vendor2" in [message] { fix_protocol { fix_message => fix_string data_dictionary_path => "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-fix_protocol-0.3.2/FIXXML/FIX44.xml" } } } ..........

Thank you.

[daino3]

@nameraj - can you provide your entire logstash config?

[nameraj]

input { beats { port => 5044 } }

filter {

fingerprint { source => [ "message" ] method => "SHA1" key => "Some super secret passphrase for uniqueness." concatenate_sources => true }

if "FIX" in [message] {

grok { patterns_dir => [ "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-patterns-core-5.3.0/patterns" ] match => ["message","%{TIMESTAMP_ISOFIX:timestamp} %{GREEDYDATA:fix_session} - >?<? %{GREEDYDATA:fix_string}"] }

if "vendor1" in [message] { fix_protocol { fix_message => fix_string data_dictionary_path => "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-fix_protocol-0.3.2/FIX.xml" } if "vendor2" in [message] { fix_protocol { fix_message => fix_string data_dictionary_path => "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-fix_protocol-0.3.2/FIXXML/FIX44.xml" }

} } output { elasticsearch { hosts => ["http://127.0.0.1:9200"] index => "fix-%{+YYYY.MM.dd}" } }

FYI I have updated plugin to 0.3.3 but my FIX.XML still pointed to old folder.

[nameraj]

i think it is related to my XML file. it doesn't have required attribute for every field but do i have to need them. When I took out every line which doesnt had required attribute, it gave me below errors.

[2017-10-23T15:55:14,923][ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Constructor invocation failed: section not found in data dictionary"}

[daino3]

Yes, this isn't an "issue", you just need help.

@cbusbey - I think you can close this.

FWIW - Do these files exist?

/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-fix_protocol-0.3.2/FIX.xml
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-fix_protocol-0.3.2/FIXXML/FIX44.xml

They're not part of the plugin project. Are you placing files in the gem path? That's not good practice. Put your data dictionaries in your project's path and provide the plugin with the absolute paths to them. Read the configuration portion of the README please.

The only sample data dictionaries provided by this project reside here

[cbusbey]

10-4 @daino3