search

Juniper / open-nti

Open Network Telemetry Collector build with open source tools
Apache License 2.0
233 stars 93 forks source link

kafka Integration - yatl error #213

Open mauser01 opened 6 years ago

mauser01 commented 6 years ago

Hello, I am trying to get the KAFKA integration working for the JTI telemetry plugin. When starting the input-jti container with the following environment variables I get the posted errors.

emit transaction failed: error_class=NameError error="uninitialized constant Yajl::Encoder\nDid you mean?  EncodingError" location="/fluentd/plugins/out_kafka.rb:127:in `parse_record'" tag="jnpr.jvision"

I have tried downgrading the version of the fluentd image from fluentd-0.12.41 to fluentd-0.12.40 as another post on the net indicated but was not successful.

Am I missing something? Thank you for your help

 input-jti:
  image: $INPUT_JTI_IMAGE_NAME:$IMAGE_TAG
  container_name: $INPUT_JTI_CONTAINER_NAME
  environment:
   - "OUTPUT_INFLUXDB=false"
   - "OUTPUT_STDOUT=true"
   - "OUTPUT_KAFKA=true"
   - "KAFKA_ADDR=172.17.0.2"
   - "KAFKA_PORT=9092"
   - "KAFKA_TOPIC=jnpr.jvision"
  ports:
   - "$LOCAL_PORT_JTI:50000/udp"
   - "$LOCAL_PORT_ANALYTICSD:50020/udp"
  volumes:
   - /etc/localtime:/etc/localtime

Errors:

2018-04-05 16:19:21 -0300 jnpr.jvision: {"device":":192.168.184.177","interface":"ge-0/0/0","egress_queue":0,"type":"egress_queue_info.packets","value":0}
2018-04-05 16:32:32 -0300 [warn]: Send exception occurred: uninitialized constant Yajl::Encoder
Did you mean?  EncodingError
2018-04-05 16:32:32 -0300 [info]: brokers has been set directly: ["172.17.0.2:9092"]
2018-04-05 16:32:32 -0300 [warn]: emit transaction failed: error_class=NameError error="uninitialized constant Yajl::Encoder\nDid you mean?  EncodingError" location="/fluentd/plugins/out_kafka.rb:127:in `parse_record'" tag="jnpr.jvision"
  2018-04-05 16:32:32 -0300 [warn]: /fluentd/plugins/out_kafka.rb:127:in `parse_record'
  2018-04-05 16:32:32 -0300 [warn]: /fluentd/plugins/out_kafka.rb:152:in `block in emit'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/event.rb:56:in `each'
  2018-04-05 16:32:32 -0300 [warn]: /fluentd/plugins/out_kafka.rb:147:in `emit'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/output.rb:42:in `next'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/buffer.rb:199:in `block in emit'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/buffer.rb:193:in `emit'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/output.rb:270:in `emit'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/output.rb:42:in `next'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/plugin/out_stdout.rb:38:in `emit'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/output.rb:42:in `next'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/plugin/out_copy.rb:78:in `emit'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/event_router.rb:90:in `emit_stream'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/event_router.rb:81:in `emit'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/plugin/socket_util.rb:155:in `block in on_message'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-juniper-telemetry-0.3.0/lib/fluent/plugin/parser_juniper_jti.rb:106:in `block (5 levels) in parse'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-juniper-telemetry-0.3.0/lib/fluent/plugin/parser_juniper_jti.rb:100:in `each'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-juniper-telemetry-0.3.0/lib/fluent/plugin/parser_juniper_jti.rb:100:in `block (4 levels) in parse'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-juniper-telemetry-0.3.0/lib/fluent/plugin/parser_juniper_jti.rb:91:in `each'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-juniper-telemetry-0.3.0/lib/fluent/plugin/parser_juniper_jti.rb:91:in `block (3 levels) in parse'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-juniper-telemetry-0.3.0/lib/fluent/plugin/parser_juniper_jti.rb:87:in `each'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-juniper-telemetry-0.3.0/lib/fluent/plugin/parser_juniper_jti.rb:87:in `block (2 levels) in parse'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-juniper-telemetry-0.3.0/lib/fluent/plugin/parser_juniper_jti.rb:64:in `each'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-juniper-telemetry-0.3.0/lib/fluent/plugin/parser_juniper_jti.rb:64:in `block in parse'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-juniper-telemetry-0.3.0/lib/fluent/plugin/parser_juniper_jti.rb:54:in `each'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluent-plugin-juniper-telemetry-0.3.0/lib/fluent/plugin/parser_juniper_jti.rb:54:in `parse'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/plugin/socket_util.rb:148:in `on_message'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/plugin/socket_util.rb:48:in `call'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/plugin/socket_util.rb:48:in `on_readable'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/io.rb:186:in `on_readable'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run_once'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run'
  2018-04-05 16:32:32 -0300 [warn]: /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.12.41/lib/fluent/plugin/socket_util.rb:139:in `run'
2018-04-05 16:32:32 -0300 [warn]: Unable to parse jnpr_interface_ext sensor, Error during processing: uninitialized constant Yajl::Encoder
Did you mean?  EncodingError
2018-04-05 16:32:32 -0300 fluent.warn: {"message":"Send exception occurred: uninitialized constant Yajl::Encoder\nDid you mean?  EncodingError"}
2018-04-05 16:32:32 -0300 fluent.info: {"message":"brokers has been set directly: [\"172.17.0.2:9092\"]"}
2018-04-05 16:32:32 -0300 fluent.warn: {"error_class":"NameError","error":"uninitialized constant Yajl::Encoder\nDid you mean?  EncodingError","location":"/fluentd/plugins/out_kafka.rb:127:in `parse_record'","tag":"jnpr.jvision","message":"emit transaction failed: error_class=NameError error=\"uninitialized constant Yajl::Encoder\\nDid you mean?  EncodingError\" location=\"/fluentd/plugins/out_kafka.rb:127:in `parse_record'\" tag=\"jnpr.jvision\""}
2018-04-05 16:32:32 -0300 fluent.warn: {"message":"Unable to parse jnpr_interface_ext sensor, Error during processing: uninitialized constant Yajl::Encoder\nDid you mean?  EncodingError"}