patant / fluent-plugin-loggly

Fluentd plugin for output to loggly
Apache License 2.0
13 stars 24 forks source link

Debugging to run this plugin successfully #10

Closed josh-padnick closed 8 years ago

josh-padnick commented 8 years ago

I've been struggling to get this plugin working successfully. I'd like to be able to debug what's going on, but I can't seem to find the plugin's log entries anywhere.

In the code, I see entries like https://github.com/patant/fluent-plugin-loggly/blob/master/lib/fluent/plugin/out_loggly.rb#L30. But in the /var/log/td-agent/td-agent.log I see none of these entries.

I'm outputting logs both to a local file and to Loggly. The local file works fine but loggly output is not working. Any thoughts on how I can resolve this?

Here's my td-agent.conf (In my actual file, I have #comments which I've removed below):

<source>
  type forward
  port 5140
  bind 0.0.0.0
</source>

<match **>
  type file
  path /tmp/sample.log
</match>

<match **>
  type                loggly_buffered
  loggly_url          https://logs-01.loggly.com/bulk/<token>/tag/fluentd
  output_include_time true  # add 'timestamp' record into log. (default: true)
  buffer_type         file
  buffer_path         /tmp/buffer
  flush_interval      2s
</match>

I've also confirmed that directly calling loggly's API via curl works fine from my box. I've also successfully installed the fluent-plugin-loggly gem by running it in a Docker container with this Dockerfile:

FROM phusion/baseimage:0.9.16

RUN apt-get update && \
    apt-get -y upgrade && \
    # Install the supported version of fluentd, "td-agent"
    curl https://packages.treasuredata.com/GPG-KEY-td-agent | apt-key add - && \
    echo "deb [arch=amd64] http://packages.treasuredata.com/2/ubuntu/trusty/ trusty contrib" > /etc/apt/sources.list.d/treasure-data.list && \
    apt-get update && \
    apt-get install -y --force-yes td-agent && \
    td-agent-gem install fluent-plugin-loggly && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

ADD files/td-agent.conf /etc/td-agent/td-agent.conf

ADD files/init.d/td-agent /etc/init.d/td-agent
RUN chmod 0700 /etc/init.d/td-agent

EXPOSE 5140

CMD ["/etc/init.d/td-agent", "start"]

At this point, I may just use the generic exec plugin, but that seems like a shame given that all of this nice work is done here. Thanks for your input!

josh-padnick commented 8 years ago

Sorry, this had nothing to do with this plugin. I didn't realize that fluentd matches the first <match> rule it can in the td-agent.conf. Since I was matching my first stanza, the one I get cared about was getting ignored.