I am upgrading ELK 5.3.1 to ELK 6.3.2 in PR environment
While re-indexing in Elastic search 6.3.2 , I am getting below error.
I have logstash running for 5.3.1 in diff port (2505) and new logstash in (9200)
I am getting this issue in indexing for this particular application. I have successfully re-indexes for others.
[2018-10-09T03:33:38,849][ERROR][logstash.pipeline ] A plugin had an unrecoverable error. Will restart this plugin.
Pipeline_id:main
Plugin: <LogStash::Inputs::Elasticsearch hosts=>["54.40.221.197:2505"], index=>"personify*", scroll=>"1m", docinfo=>true, id=>"82a7e777e1a4659871950c5a0714a8f237db44a6cc23031491cb4b50b8a5b41a", enable_metric=>true, codec=><LogStash::Codecs::JSON id=>"json_87b12647-44a7-4ada-b2df-1cfe95d677e0", enable_metric=>true, charset=>"UTF-8">, query=>"{ \"sort\": [ \"_doc\" ] }", size=>1000, docinfo_target=>"@metadata", docinfo_fields=>["_index", "_type", "_id"], ssl=>false>
Error: Net::ReadTimeout
Exception: Faraday::TimeoutError
Stack: uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/protocol.rb:158:in rbuf_fill' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/protocol.rb:136:inreaduntil'
below is my template for indexing:-
input {
We read from the "old" cluster
elasticsearch {
hosts => [ "54.40.221.197:2505" ]
index => "personify*"
I am upgrading ELK 5.3.1 to ELK 6.3.2 in PR environment While re-indexing in Elastic search 6.3.2 , I am getting below error. I have logstash running for 5.3.1 in diff port (2505) and new logstash in (9200) I am getting this issue in indexing for this particular application. I have successfully re-indexes for others.
[2018-10-09T03:33:38,849][ERROR][logstash.pipeline ] A plugin had an unrecoverable error. Will restart this plugin. Pipeline_id:main Plugin: <LogStash::Inputs::Elasticsearch hosts=>["54.40.221.197:2505"], index=>"personify*", scroll=>"1m", docinfo=>true, id=>"82a7e777e1a4659871950c5a0714a8f237db44a6cc23031491cb4b50b8a5b41a", enable_metric=>true, codec=><LogStash::Codecs::JSON id=>"json_87b12647-44a7-4ada-b2df-1cfe95d677e0", enable_metric=>true, charset=>"UTF-8">, query=>"{ \"sort\": [ \"_doc\" ] }", size=>1000, docinfo_target=>"@metadata", docinfo_fields=>["_index", "_type", "_id"], ssl=>false> Error: Net::ReadTimeout Exception: Faraday::TimeoutError Stack: uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/protocol.rb:158:in
rbuf_fill' uri:classloader:/META-INF/jruby.home/lib/ruby/stdlib/net/protocol.rb:136:in
readuntil'below is my template for indexing:- input {
We read from the "old" cluster
elasticsearch { hosts => [ "54.40.221.197:2505" ] index => "personify*"
size => 500
query => '{ "query": { "match": { "topic": "TrendProd" } } , "sort": [ "_doc" ]}'
} }
output {
We write to the "new" cluster
elasticsearch { hosts => [ "54.40.221.197:9200" ] document_type => "doc" index => "sqs-personify-%{+YYYY.MM}" }
We print dots to see it in action
stdout {
codec => "json_lines"
} }