elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
14.2k stars 3.5k forks source link

Enhancement Request: Provide Proxy Capability For Logstash X-PackMonitoring Parameters #9560

Open MorrieAtElastic opened 6 years ago

MorrieAtElastic commented 6 years ago

This is the public version of https://github.com/elastic/enhancements/issues/4107

This feature request is to enable proxy addressing for the logstash "xpack.monitoring.elasticsearch.url" parameter.

-This enhancement speaks to a problem which customers may encounter if their connections between logstash and elasticsearch are blocked by some kind of firewall. In this specific case, the elasticsearch cluster is located in the Elastic Cloud while logstash resides in the Azure Cloud. All communications between processes executing inside Azure and processes residing outside of Azure require a proxy. When sending event data from logstash to his Elastic Cloud cluster, a proxy can be designated from with the logstash output plugin:

elasticsearch { hosts => "" user => "someuser" password => "somepassword" manage_template => false index => "%{[@metadata][beat]}-%{[fields][document_type]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" proxy => "" }

However, with respect to X-Pack monitoring configuration, there is no such proxy field available to work with "xpack.monitoring.elasticsearch.url". It is therefore not possible to send monitoring data to logstash when using such a configuration.

This is a reasonable request and presumably there are many other customers with the same need: any configuration where logstash operates behind some kind of IP-filtering firewall is going to block customer's ability to provide monitoring data from logstash to the appropriate monitoring cluster. So the ability to provide a proxy would be of value to all customers working in such configurations.

roaksoax commented 4 years ago

This issue has now been fixed and will be available in Logstash 7.8. Please refer to the attached PR's for more information.