search

DataDog / fluent-plugin-datadog

Fluentd output plugin for Datadog: https://www.datadog.com
Apache License 2.0
39 stars 26 forks source link

Exception when logs are sent to Datadog (wrong number of arguments (given 1, expected 0)) #37

Closed sabarivasan closed 6 months ago

sabarivasan commented 4 years ago

Describe what happened: Exception when logs are sent to datadog :

  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/net-http-persistent-3.1.0/lib/net/http/persistent.rb:530:in `initialize'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-datadog-0.12.0/lib/fluent/plugin/out_datadog.rb:305:in `new'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-datadog-0.12.0/lib/fluent/plugin/out_datadog.rb:305:in `initialize'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-datadog-0.12.0/lib/fluent/plugin/out_datadog.rb:261:in `new'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-datadog-0.12.0/lib/fluent/plugin/out_datadog.rb:261:in `new_client'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-datadog-0.12.0/lib/fluent/plugin/out_datadog.rb:91:in `start'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/lib/fluent/root_agent.rb:200:in `block in start'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/lib/fluent/root_agent.rb:189:in `block (2 levels) in lifecycle'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/lib/fluent/root_agent.rb:188:in `each'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/lib/fluent/root_agent.rb:188:in `block in lifecycle'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/lib/fluent/root_agent.rb:175:in `each'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/lib/fluent/root_agent.rb:175:in `lifecycle'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/lib/fluent/root_agent.rb:199:in `start'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/lib/fluent/engine.rb:248:in `start'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/lib/fluent/engine.rb:147:in `run'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/lib/fluent/supervisor.rb:596:in `block in run_worker'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/lib/fluent/supervisor.rb:831:in `main_process'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/lib/fluent/supervisor.rb:590:in `run_worker'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/lib/fluent/command/fluentd.rb:338:in `<top (required)>'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/2.5.0/rubygems/core_ext/kernel_require.rb:59:in `require'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/lib/ruby/gems/2.5.0/gems/fluentd-1.9.3/bin/fluentd:8:in `<top (required)>'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/bin/fluentd:23:in `load'
  2020-03-14 13:16:55 +0000 [error]: #0 /usr/bin/fluentd:23:in `<main>'
2020-03-14 13:16:55 +0000 [error]: #0 unexpected error error_class=ArgumentError error="wrong number of arguments (given 1, expected 0)"

Describe what you expected: Logs to be sent to Datadog

**Steps to repro

duce the issue:** Docker image with following versions: Ruby version: 2.5.7 fluentd version: 1.9.3 fluent-plugin-datadog version: 0.12.0

(22/27) Installing ruby-libs (2.5.7-r0)
(23/27) Installing ruby (2.5.7-r0)
(24/27) Installing ruby-etc (2.5.7-r0)
(25/27) Installing ruby-irb (2.5.7-r0)
(26/27) Installing ruby-webrick (2.5.7-r0)

Additional environment details (Operating System, Cloud provider, etc): alpine 3.9. Dockerfile attached

sabarivasan commented 4 years ago

Dockerfile.txt

tobyfoo commented 4 years ago

Same here, on version 0.12.0. Doesn't happen when installing previous version (gem -v 0.11.1), using this as a workaround, hth.

gaetan-deputier commented 4 years ago

Hey! Thanks for the report. We're going to take a look this week and release a fix.

gaetan-deputier commented 4 years ago

Hey @sabarivasan I've ran fluentd with the datadog output plugin on 0.12.0 with the following dockerfile


RUN apk update \
 && apk add --no-cache \
        ca-certificates \
        ruby ruby-irb ruby-etc ruby-webrick \
        curl jq \
        tini \
 && apk add --no-cache --virtual .build-deps \
        build-base linux-headers \
        ruby-dev gnupg \
 && echo 'gem: --no-document' >> /etc/gemrc \
 && gem install oj -v 3.8.1 \
 && gem install json -v 2.3.0 \
 && gem install async-http -v 0.50.0 \
 && gem install ext_monitor -v 0.1.1 \
 && gem install fluentd -v 1.9.3 \
 && gem install bigdecimal -v 1.4.4 \
 && gem install fluent-plugin-datadog -v 0.12.0 \
 && apk del .build-deps \
 && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem

COPY fluent.conf /etc/fluent/fluent.conf

RUN addgroup -S fluent && adduser -S -g fluent fluent \
    && mkdir -p /fluentd/log \
    && mkdir -p /fluentd/etc /fluentd/plugins \
    && chown -R fluent /fluentd && chgrp -R fluent /fluentd \
    && chown -R fluent:fluent /usr/lib/ruby/gems/2.5.0/gems/fluent-plugin-datadog* \

EXPOSE 24224 5140
ENTRYPOINT ["tini", "--"]
CMD ["fluentd"]

which is very similar to the one you proposed. I did not ran onto your issue though. I have tried as well to add the Splunk collector that you have and did not have any issue.

I've upgraded our net.persistent.http to 3.1.0 in this PR: https://github.com/DataDog/fluent-plugin-datadog/pull/40 and updated the init by arg name which should hopefully cover your case.

I'm curious about the plugin configuration as well if you had that handy that would help!

demonccc commented 4 years ago

Hi @gaetan-deputier , I'm having the same error as @sabarivasan 

2020-05-14 02:56:24 +0000 [error]: #0 unexpected error error_class=ArgumentError error="wrong number of arguments (given 1, expected 0)"
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/net-http-persistent-3.1.0/lib/net/http/persistent.rb:530:in `initialize'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluent-plugin-datadog-0.12.0/lib/fluent/plugin/out_datadog.rb:305:in `new'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluent-plugin-datadog-0.12.0/lib/fluent/plugin/out_datadog.rb:305:in `initialize'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluent-plugin-datadog-0.12.0/lib/fluent/plugin/out_datadog.rb:261:in `new'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluent-plugin-datadog-0.12.0/lib/fluent/plugin/out_datadog.rb:261:in `new_client'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluent-plugin-datadog-0.12.0/lib/fluent/plugin/out_datadog.rb:91:in `start'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/root_agent.rb:200:in `block in start'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/root_agent.rb:179:in `block (2 levels) in lifecycle'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/agent.rb:121:in `block (2 levels) in lifecycle'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/agent.rb:120:in `each'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/agent.rb:120:in `block in lifecycle'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/agent.rb:113:in `each'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/agent.rb:113:in `lifecycle'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/root_agent.rb:178:in `block in lifecycle'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/root_agent.rb:175:in `each'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/root_agent.rb:175:in `lifecycle'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/root_agent.rb:199:in `start'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/engine.rb:248:in `start'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/engine.rb:147:in `run'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/supervisor.rb:600:in `block in run_worker'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/supervisor.rb:837:in `main_process'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/supervisor.rb:594:in `run_worker'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/lib/fluent/command/fluentd.rb:350:in `<top (required)>'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/gems/fluentd-1.10.2/bin/fluentd:8:in `<top (required)>'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/bin/fluentd:23:in `load'
  2020-05-14 02:56:24 +0000 [error]: #0 /usr/local/bundle/bin/fluentd:23:in `<main>'
2020-05-14 02:56:24 +0000 [error]: #0 unexpected error error_class=ArgumentError error="wrong number of arguments (given 1, expected 0)"
  2020-05-14 02:56:24 +0000 [error]: #0 suppressed same stacktrace

This is my Dockerfile:

FROM fluent/fluentd:v1.10-debian-1

# Use root account to use apt
USER root

# below RUN includes plugin as examples elasticsearch is not required
# you may customize including plugins as you wish
RUN buildDeps="sudo make gcc g++ libc-dev" \
 && apt-get update \
 && apt-get install -y --no-install-recommends $buildDeps \
 && apt-get install -y --no-install-recommends wget smitools \
 && gem install fluent-plugin-elasticsearch \
 && gem install fluent-plugin-prometheus \
 && gem install fluent-plugin-rewrite-tag-filter \
 && gem install fluent-plugin-record-modifier \
 && gem install fluent-plugin-multi-format-parser \
 && gem install fluent-plugin-splunk-hec \
 && gem install influxdb-plugin-fluent \
 && gem install fluent-plugin-datadog \
 && gem install snmp \
 && gem sources --clear-all \
 && wget https://raw.githubusercontent.com/Azure/fluent-plugin-snmptrapalert/master/lib/fluent/plugin/in_snmptrapalert.rb -O /fluentd/plugins/in_snmptrapalert.rb \
 && wget https://raw.githubusercontent.com/cosmo0920/fluent-plugin-statsd/migrate-v1/lib/fluent/plugin/out_statsd.rb -O /fluentd/plugins/out_statsd.rb \
 && SUDO_FORCE_REMOVE=yes \
    apt-get purge -y --auto-remove \
                  -o APT::AutoRemove::RecommendsImportant=false \
                  $buildDeps \
 && rm -rf /var/lib/apt/lists/* \
 && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem

USER fluent

Regards

jszwedko commented 6 months ago

I'm not able to reproduce this with the indicated fluentd and plugin versions:

fluentd version: 1.9.3
fluent-plugin-datadog version: 0.12.0

If anyone is still running into this, please share details about your environment and fluentd configuration file.