fluent-plugins-nursery / fluent-plugin-bigquery

Other
275 stars 98 forks source link

I can't output data to Bigqueiry when the contents cotains Chinese #182

Open MinghuiChen opened 5 years ago

MinghuiChen commented 5 years ago
I can't output data to Bigqueiry when the contents cotains Chinese  in WINDOWS。
Errors like these:

2019-09-19 13:17:25 +0800 [debug]: #0 taking back chunk for errors. chunk="592e1119616f5807079b362bfb2383f9" 2019-09-19 13:17:25 +0800 [warn]: #0 failed to flush the buffer. retry_time=0 next_retry_seconds=2019-09-19 13:17:26 +0800 chunk="592e1119616f5807079b362bfb2383f9" error_class=Encoding::CompatibilityError error="incompatible character encodings: ASCII-8BIT and UTF-8" 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/http.rb:510:in dump' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/http.rb:962:indump' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/session.rb:510:in block in query' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/2.4.0/timeout.rb:93:inblock in timeout' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/2.4.0/timeout.rb:103:in timeout' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/session.rb:508:inquery' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient/session.rb:174:in query' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1240:indo_get_block' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1017:in block in do_request' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1131:inprotect_keep_alive_disconnected' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1012:in do_request' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:1102:infollow_redirect' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/httpclient-2.8.2.4/lib/httpclient.rb:852:in request' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/http_command.rb:302:inexecute_once' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/http_command.rb:113:in block (2 levels) in execute' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:61:inblock in retriable' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:56:in times' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:56:inretriable' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/http_command.rb:110:in block in execute' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:61:inblock in retriable' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:56:in times' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/retriable-3.1.2/lib/retriable.rb:56:inretriable' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/http_command.rb:102:in execute' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/lib/google/apis/core/base_service.rb:360:inexecute_or_queue_command' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/google-api-client-0.30.3/generated/google/apis/bigquery_v2/service.rb:1058:in insert_all_table_data' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-bigquery-2.2.0/lib/fluent/plugin/bigquery/writer.rb:85:ininsert_rows' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-bigquery-2.2.0/lib/fluent/plugin/out_bigquery_insert.rb:99:in insert' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluent-plugin-bigquery-2.2.0/lib/fluent/plugin/out_bigquery_insert.rb:95:inwrite' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin/output.rb:1122:in try_flush' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin/output.rb:1428:inflush_thread_run' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin/output.rb:458:in block (2 levels) in start' 2019-09-19 13:17:25 +0800 [warn]: #0 C:/opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/fluentd-1.7.0/lib/fluent/plugin_helper/thread.rb:78:inblock in thread_create'

This error only happened in Windows.Linux is OK. Who can help me??