elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
14.18k stars 3.5k forks source link

Connectivity between logstsash server and elasticsearch #9694

Open himmatb opened 6 years ago

himmatb commented 6 years ago

Component version's :

Elasticsearch version: 2.4.4 Searchguard version: 2.4.4 Logstash version: 5.6.9

OS: RHEL 7

I choose this logstash 5.6.9 version after confirming with compatibilty with elasticsearch version https://www.elastic.co/support/matrix#matrix_compatibility

Below is logstash configuration:

output {
  elasticsearch {
    hosts => ["logging-es:9200"]
    cacert => '/etc/logstash/conf.d/keys/admin-ca.crt'
    ssl => true
    ssl_certificate_verification => false
    index => "logstash-%{+YYYY.MM.dd}"
    manage_template => false
    document_type => "%{[@metadata][type]}"
   }
  stdout { codec => rubydebug }
}

I am getting below error :

11:45:44.093 [Ruby-0-Thread-5: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-7.4.2-java/lib/logstash/outputs/elasticsearch/http_client/pool.rb:228] WARN logstash.outputs.elasticsearch - Attempted to resurrect connection to dead ES instance, but got an error. {:url=>"http://logging-es:9200/", :error_type=>LogStash::Outputs::ElasticSearch::HttpClient::Pool::HostUnreachableError, :error=>"Elasticsearch Unreachable: [http://logging-es:9200/][Manticore::ClientProtocolException] logging-es:9200 failed to respond"}

himmatb commented 6 years ago

Please help me on this..

jsvd commented 6 years ago

can you try with hosts => ["https://logging-es:9200"] ?

himmatb commented 6 years ago

Getting same error.. I am not sure whether this issue with elasticsearch logstash outputplugin or with my configuration.. I tried with truststore configuration.. Getting same error.. Please help me on this..

jsvd commented 6 years ago

can you confirm you can connect to ES with curl https://logging-es:9200 from that machine?

himmatb commented 6 years ago

I have elasticsearch configuration which using TLSv1.2 SSL protocol so Logstash must use TLSv1.2 or later to talk to Elasticsearch .I am not sure where to make configuration to make Logstsash output plugin to send logs using TLSv1.2 protocol.

how can I achieve this .. Please help me to resolve this ..

himmatb commented 6 years ago

Please let me know if you have aby workarounds to fix this issue..

himmatb commented 6 years ago

Is there anyway I can use certificate like below to connect with elasticsaerch:

 ssl_certificate => "/etc/logstash/ssl/logstash.crt"
  ssl_key => "/etc/logstash/ssl/logstash.key"

and ca certificate ?

Jymit commented 6 years ago

@himmatb I believe this is one ideally raised on the discuss forum. As my first note would be to review the SG LS v5 docs, for example here