search

fangli / fluent-plugin-influxdb

A buffered output plugin for fluentd and InfluxDB
MIT License
111 stars 65 forks source link

Undefined method "nsec" #70

Closed stvnwrgs closed 7 years ago

stvnwrgs commented 7 years ago

When trying to run the fluent-plugin in 1.0.0-rc1 i get the following error:

Error log:

2017-02-04 21:12:14 +0000 [warn]: emit transaction failed: error_class=NoMethodError error="undefined method `nsec' for 1486242734:Fixnum\nDid you mean?  inspect" tag="metric.docker.upbeat_allen"
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluent-plugin-influxdb-1.0.0.rc1/lib/fluent/plugin/out_influxdb.rb:217:in `precision_time'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluent-plugin-influxdb-1.0.0.rc1/lib/fluent/plugin/out_influxdb.rb:108:in `format'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/plugin/output.rb:876:in `block in handle_stream_simple'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/event.rb:193:in `block in each'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/event.rb:192:in `each'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/event.rb:192:in `each'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/plugin/output.rb:875:in `handle_stream_simple'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/plugin/output.rb:770:in `execute_chunking'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/plugin/output.rb:707:in `emit_buffered'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/event_router.rb:153:in `emit_events'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/event_router.rb:90:in `emit_stream'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/event_router.rb:81:in `emit'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluent-plugin-grep-0.3.4/lib/fluent/plugin/out_grep.rb:77:in `block (2 levels) in emit'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluent-plugin-grep-0.3.4/lib/fluent/plugin/out_grep.rb:70:in `catch'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluent-plugin-grep-0.3.4/lib/fluent/plugin/out_grep.rb:70:in `block in emit'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/event.rb:107:in `each'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluent-plugin-grep-0.3.4/lib/fluent/plugin/out_grep.rb:69:in `emit'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/compat/output.rb:162:in `process'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/plugin/output.rb:696:in `emit_sync'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/event_router.rb:90:in `emit_stream'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/event_router.rb:81:in `emit'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/plugin/in_forward.rb:377:in `on_message'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/plugin/in_forward.rb:231:in `block in handle_connection'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/plugin/in_forward.rb:272:in `block (3 levels) in read_messages'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/plugin/in_forward.rb:271:in `feed_each'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/plugin/in_forward.rb:271:in `block (2 levels) in read_messages'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/plugin/in_forward.rb:280:in `block in read_messages'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/plugin/in_forward.rb:560:in `on_read'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/cool.io-1.4.6/lib/cool.io/io.rb:123:in `on_readable'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/cool.io-1.4.6/lib/cool.io/io.rb:186:in `on_readable'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/cool.io-1.4.6/lib/cool.io/loop.rb:88:in `run_once'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/cool.io-1.4.6/lib/cool.io/loop.rb:88:in `run'
  2017-02-04 21:12:14 +0000 [warn]: /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluentd-0.14.8/lib/fluent/plugin/in_forward.rb:190:in `run'
2017-02-04 21:12:14 +0000 [warn]: out_grep: NoMethodError undefined method `nsec' for 1486242734:Fixnum
Did you mean?  inspect /opt/rh/rh-ruby23/root/usr/local/share/gems/gems/fluent-plugin-influxdb-1.0.0.rc1/lib/fluent/plugin/out_influxdb.rb:217:in `precision_time'

Fluentd Dockerfile

FROM centos:7

MAINTAINER Jimmi Dyson <jimmidyson@gmail.com>

ENTRYPOINT ["/start-fluentd"]

ENV LD_LIBRARY_PATH /opt/rh/rh-ruby23/root/usr/lib64
ENV FLUENTD_VERSION 0.14.8
ENV ELASTICSEARCH_HOST es-logging.default.svc

RUN yum update -y && \
    yum install -y centos-release-scl-rh && \
    yum install -y scl-utils make gcc bzip2 rh-ruby23 rh-ruby23-ruby-devel && \
    yum clean all

RUN scl enable rh-ruby23 'gem update --system --no-document' && \
    scl enable rh-ruby23 'gem install --no-document json_pure jemalloc' && \
    scl enable rh-ruby23 "gem install --no-document fluentd -v ${FLUENTD_VERSION}" && \
    ln -s /opt/rh/rh-ruby23/root/usr/local/bin/* /usr/bin

ENV ELASTICSEARCH_HOST es-logging.default.svc
RUN touch /var/lib/rpm/* && yum install -y gcc-c++ && yum clean all

RUN scl enable rh-ruby23 'gem install --no-document fluent-plugin-elasticsearch -v 1.9.2' && \
    scl enable rh-ruby23 'gem install fluent-plugin-influxdb -v 1.0.0.rc1 --no-document' && \
    scl enable rh-ruby23 'gem install fluent-plugin-parser' && \
    scl enable rh-ruby23 'gem install fluent-plugin-grep' && \
    scl enable rh-ruby23 'gem cleanup fluentd'

ADD start-fluentd /start-fluentd

Fluentd config:

<source>
  type forward
  port 24224
  bind 0.0.0.0
</source>

<match docker.**>
  @type grep
  regexp1 log resource_request
  add_tag_prefix metric
</match>

<match metric.docker.**>
  @type influxdb
  measurement measurement
  host  10.93.1.21
  port  8086
  dbname db1
  user  root
  password  somepassword
  use_ssl false
  time_precision s
  tag_keys ["key1", "key2"]
  sequence_tag _seq
</match>

<match metric.docker.**>
  @type stdout
</match># input plugin that exports metrics

Stdout:

{"measurement":"resource_request","tags":{"name":"asd","value":"asd","key1":"test"},"values":{"response_time":1,"status":200}}
sabueso commented 7 years ago

Maybe I'm wrong, but, did you try to defining a key:value as "time" with timestamp value? https://github.com/fangli/fluent-plugin-influxdb/issues/67

repeatedly commented 7 years ago

Ah, sorry. I missed this issue. I just released v1.0.0.rc2 for fixing this problem. Could you try it?

openbsod commented 7 years ago

Hi. Sorry for reopening.. same there with fluent-plugin-netflow and fluent-plugin-influxdb. Netflow v9.

type netflow tag netflow.event

optional parameters

bind xx.xx.195.182 port 9996 cache_ttl 6000

definitions /root/conf/netflow_fields.yaml

@type influxdb host localhost port 8086 user fluent password fluent dbname traffic use_ssl false # time_precision ns sequence_tag _seq

ruby

repeatedly commented 7 years ago

@openbsod This is a bug of netflow plugin. Try fluent-plugin-netflow v1.0.0.rc2.

openbsod commented 7 years ago

@repeatedly Thank you. Gem cleanup + reboot did the trick. Now everything in its place:

influxx