search

AliyunContainerService / log-pilot

Collect logs for docker containers
https://yq.aliyun.com/articles/674327
Apache License 2.0
1.43k stars 400 forks source link

failed to flush the buffer. retry_time=0 next_retry_seconds=2017-11-12 12:55:43 +0000 chunk="55dc8aba9dbc2342f98607c506e7f654" error_class=Encoding::UndefinedConversionError error="\"\\xE5\" from ASCII-8BIT to UTF-8" #66

Closed haqiaolong closed 6 years ago

haqiaolong commented 7 years ago

hi 我用 graylog去收集日志 全英文的没问题 但是包含中文 就报错，报错如下： 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/gelf-3.0.0/lib/gelf/notifier.rb:160:in encode' 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/gelf-3.0.0/lib/gelf/notifier.rb:160:into_json' 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/gelf-3.0.0/lib/gelf/notifier.rb:160:in notify_with_level!' 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/gelf-3.0.0/lib/gelf/notifier.rb:119:innotify!' 2017-11-12 12:55:42 +0000 [warn]: #0 /etc/fluentd/plugins/out_gelf.rb:119:in block in write' 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.22/lib/fluent/event.rb:323:ineach' 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.22/lib/fluent/event.rb:323:in block in each' 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.22/lib/fluent/plugin/buffer/memory_chunk.rb:80:inopen' 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.22/lib/fluent/plugin/buffer/memory_chunk.rb:80:in open' 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.22/lib/fluent/event.rb:322:ineach' 2017-11-12 12:55:42 +0000 [warn]: #0 /etc/fluentd/plugins/out_gelf.rb:118:in write' 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.22/lib/fluent/compat/output.rb:131:inwrite' 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.22/lib/fluent/plugin/output.rb:1089:in try_flush' 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.22/lib/fluent/plugin/output.rb:1314:inflush_thread_run' 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.22/lib/fluent/plugin/output.rb:439:in block (2 levels) in start' 2017-11-12 12:55:42 +0000 [warn]: #0 /usr/lib/ruby/gems/2.3.0/gems/fluentd-0.14.22/lib/fluent/plugin_helper/thread.rb:78:inblock in thread_create' 2017-11-12 12:55:43 +0000 [warn]: #0 failed to flush the buffer. retry_time=0 next_retry_seconds=2017-11-12 12:55:43 +0000 chunk="55dc8aba9dbc2342f98607c506e7f654" error_class=Encoding::UndefinedConversionError error="\"\xE5\" from ASCII-8BIT to UTF-8" 2017-11-12 12:55:43 +0000 [warn]: #0 suppressed same stacktrace 2017-11-12 12:55:44 +0000 [warn]: #0 failed to flush the buffer. retry_time=1 next_retry_seconds=2017-11-12 12:55:44 +0000 chunk="55dc8aba9dbc2342f98607c506e7f654" error_class=Encoding::UndefinedConversionError error="\"\xE5\" from ASCII-8BIT to UTF-8" 2017-11-12 12:55:44 +0000 [warn]: #0 suppressed same stacktrace 2017-11-12 12:55:46 +0000 [warn]: #0 failed to flush the buffer. retry_time=2 next_retry_seconds=2017-11-12 12:55:46 +0000 chunk="55dc8aba9dbc2342f98607c506e7f654" error_class=Encoding::UndefinedConversionError error="\"\xE5\" from ASCII-8BIT to UTF-8" 2017-11-12 12:55:46 +0000 [warn]: #0 suppressed same stacktrace 2017-11-12 12:55:50 +0000 [warn]: #0 failed to flush the buffer. retry_time=3 next_retry_seconds=2017-11-12 12:55:50 +0000 chunk="55dc8aba9dbc2342f98607c506e7f654" error_class=Encoding::UndefinedConversionError error="\"\xE5\" from ASCII-8BIT to UTF-8" 2017-11-12 12:55:50 +0000 [warn]: #0 suppressed same stacktrace 2017-11-12 12:55:58 +0000 [warn]: #0 failed to flush the buffer. retry_time=4 next_retry_seconds=2017-11-12 12:55:58 +0000 chunk="55dc8aba9dbc2342f98607c506e7f654" error_class=Encoding::UndefinedConversionError error="\"\xE5\" from ASCII-8BIT to UTF-8" 2017-11-12 12:55:58 +0000 [warn]: #0 suppressed same stacktrace ^C^C^C^C2017-11-12 12:56:15 +0000 [warn]: #0 failed to flush the buffer. retry_time=5 next_retry_seconds=2017-11-12 12:56:15 +0000 chunk="55dc8aba9dbc2342f98607c506e7f654" error_class=Encoding::UndefinedConversionError error="\"\xE5\" from ASCII-8BIT to UTF-8" 2017-11-12 12:56:15 +0000 [warn]: #0 suppressed same stacktrace 2017-11-12 12:56:50 +0000 [warn]: #0 failed to flush the buffer. retry_time=6 next_retry_seconds=2017-11-12 12:56:50 +0000 chunk="55dc8aba9dbc2342f98607c506e7f654" error_class=Encoding::UndefinedConversionError error="\"\xE5\" from ASCII-8BIT to UTF-8"

carlvine500 commented 6 years ago

we came across the same problem . and then fixed with latest fluentd graylog gelf plugin

resolve it simplely as following : step1: generate latest fluentd graylog gelf : gelf-latest.gem step2: generate a docker image

FROM registry.cn-hangzhou.aliyuncs.com/acs-sample/log-pilot:latest
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories
COPY plugins/ /etc/fluentd/plugins/
RUN apk update && \
    apk add python && \
    apk add ruby-json ruby-irb && \
    apk add build-base ruby-dev && \
    apk add python && \
    apk add lsof && \
    apk add ca-certificates wget && \
    gem install fluentd -v 1.1.3 --no-ri --no-rdoc && \
    gem install --local /etc/fluentd/plugins/gelf-latest.gem --no-ri --no-rdoc && \
    apk del build-base ruby-dev && \
    rm -rf /root/.gem

COPY ./fluentd.tpl /pilot/fluentd.tpl
loveyang2012 commented 4 years ago

` step1: generate latest fluentd graylog gelf : gelf-latest.gem

` where can i get this file "gelf-latest.gem" ，or can you send me by email 905883566@qq.com? thank you very much！

carlvine500 commented 4 years ago

gelf-3.0.1.gem.gz just rename gelf-3.0.1.gem.gz to gelf-3.0.1.gem . @loveyang2012 can you download it from this comment?