Open MinghuiChen opened 5 years ago
I can't output data to Bigqueiry when the contents cotains Chinese in WINDOWS。 Errors like these:
2019-09-19 13:17:25 +0800 [debug]: #0 taking back chunk for errors. chunk="592e1119616f5807079b362bfb2383f9" 2019-09-19 13:17:25 +0800 [warn]: #0 failed to flush the buffer. retry_time=0 next_retry_seconds=2019-09-19 13:17:26 +0800 chunk="592e1119616f5807079b362bfb2383f9" error_class=Encoding::CompatibilityError error="incompatible character encodings: ASCII-8BIT and UTF-8" 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/http.rb:510:in dump' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/http.rb:962:indump' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/session.rb:510:in block in query' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/2.4.0/timeout.rb:93:inblock in timeout' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/2.4.0/timeout.rb:103:in timeout' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/session.rb:508:inquery' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/session.rb:174:in query' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1240:indo_get_block' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1017:in block in do_request' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1131:inprotect_keep_alive_disconnected' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1012:in do_request' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1102:infollow_redirect' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:852:in request' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/http_command.rb:302:inexecute_once' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/http_command.rb:113:in block (2 levels) in execute' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:61:inblock in retriable' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:56:in times' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:56:inretriable' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/http_command.rb:110:in block in execute' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:61:inblock in retriable' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:56:in times' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:56:inretriable' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/http_command.rb:102:in execute' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/base_service.rb:360:inexecute_or_queue_command' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/generated/google/apis/bigquery_v2/service.rb:1058:in insert_all_table_data' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-bigquery-2.2.0/lib/fluent/plugin/bigquery/writer.rb:85:ininsert_rows' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-bigquery-2.2.0/lib/fluent/plugin/out_bigquery_insert.rb:99:in insert' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-bigquery-2.2.0/lib/fluent/plugin/out_bigquery_insert.rb:95:inwrite' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin/output.rb:1122:in try_flush' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin/output.rb:1428:inflush_thread_run' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin/output.rb:458:in block (2 levels) in start' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin_helper/thread.rb:78:inblock in thread_create'
dump' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/http.rb:962:in
block in query' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/2.4.0/timeout.rb:93:in
timeout' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/session.rb:508:in
query' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1240:in
block in do_request' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1131:in
do_request' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1102:in
request' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/http_command.rb:302:in
block (2 levels) in execute' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:61:in
times' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:56:in
block in execute' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:61:in
execute' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/base_service.rb:360:in
insert_all_table_data' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-bigquery-2.2.0/lib/fluent/plugin/bigquery/writer.rb:85:in
insert' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-bigquery-2.2.0/lib/fluent/plugin/out_bigquery_insert.rb:95:in
try_flush' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin/output.rb:1428:in
block (2 levels) in start' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin_helper/thread.rb:78:in
This error only happened in Windows.Linux is OK. Who can help me??
2019-09-19 13:17:25 +0800 [debug]: #0 taking back chunk for errors. chunk="592e1119616f5807079b362bfb2383f9" 2019-09-19 13:17:25 +0800 [warn]: #0 failed to flush the buffer. retry_time=0 next_retry_seconds=2019-09-19 13:17:26 +0800 chunk="592e1119616f5807079b362bfb2383f9" error_class=Encoding::CompatibilityError error="incompatible character encodings: ASCII-8BIT and UTF-8" 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/http.rb:510:in
dump' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/http.rb:962:in
dump' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/session.rb:510:inblock in query' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/2.4.0/timeout.rb:93:in
block in timeout' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/2.4.0/timeout.rb:103:intimeout' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/session.rb:508:in
query' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/session.rb:174:inquery' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1240:in
do_get_block' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1017:inblock in do_request' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1131:in
protect_keep_alive_disconnected' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1012:indo_request' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1102:in
follow_redirect' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:852:inrequest' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/http_command.rb:302:in
execute_once' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/http_command.rb:113:inblock (2 levels) in execute' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:61:in
block in retriable' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:56:intimes' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:56:in
retriable' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/http_command.rb:110:inblock in execute' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:61:in
block in retriable' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:56:intimes' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:56:in
retriable' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/http_command.rb:102:inexecute' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/base_service.rb:360:in
execute_or_queue_command' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/generated/google/apis/bigquery_v2/service.rb:1058:ininsert_all_table_data' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-bigquery-2.2.0/lib/fluent/plugin/bigquery/writer.rb:85:in
insert_rows' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-bigquery-2.2.0/lib/fluent/plugin/out_bigquery_insert.rb:99:ininsert' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-bigquery-2.2.0/lib/fluent/plugin/out_bigquery_insert.rb:95:in
write' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin/output.rb:1122:intry_flush' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin/output.rb:1428:in
flush_thread_run' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin/output.rb:458:inblock (2 levels) in start' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin_helper/thread.rb:78:in
block in thread_create'This error only happened in Windows.Linux is OK. Who can help me??