search

elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
12.06k stars 4.89k forks source link

Metricbeat xpack monitoring cannot deal with trailing slash on the url #27798

Closed philippkahr closed 3 months ago

philippkahr commented 2 years ago

Beats: 7.14.1 MacOS 11.5.2 and Windows 10.

Steps to reproduce:

  1. Start a cluster with docker
  2. Configure metricbeat to use the elasticsearch-xpack.yml module
  3. For hosts: ["http://localhost:9200"] change to hosts: ["http://localhost:9200/"] so it features a trailing /.
  4. Observe the 404 and 400 errors coming in.
2021-09-08T09:19:46.039+0200    INFO    module/wrapper.go:259   Error fetching data for metricset elasticsearch.shard: HTTP error 400 in : 400 Bad Request
2021-09-08T09:19:46.040+0200    INFO    module/wrapper.go:259   Error fetching data for metricset elasticsearch.cluster_stats: HTTP error 404 in : 404 Not Found
2021-09-08T09:19:46.047+0200    INFO    module/wrapper.go:259   Error fetching data for metricset elasticsearch.ml_job: HTTP error 400 in : 400 Bad Request
2021-09-08T09:19:46.047+0200    INFO    module/wrapper.go:259   Error fetching data for metricset elasticsearch.enrich: HTTP error 404 in : 404 Not Found

Steps to fix:

  1. Remove the trailing / slash
  2. See monitoring data coming in.

Expected behaviour:

philippkahr commented 2 years ago

If I am not mistaken e.g. the line https://github.com/elastic/beats/blob/76974f88af08920b153e21d57c177294a15ab095/metricbeat/module/elasticsearch/elasticsearch.go#L163 automatically appends /_nodes to the settings from the hosts e.g. http://localhost:9200//_nodes whilst the https://github.com/elastic/beats/blob/76974f88af08920b153e21d57c177294a15ab095/metricbeat/module/elasticsearch/elasticsearch.go#L183 appends _cluster which results in a correct http://localhost:9200/_cluster.

I guess we need to parse the hosts properly and verify if appending a / is necessary.

elasticmachine commented 2 years ago

Pinging @elastic/stack-monitoring (Stack monitoring)

elasticmachine commented 2 years ago

Pinging @elastic/integrations (Team:Integrations)

jsoriano commented 2 years ago

May be related or caused by https://github.com/elastic/beats/issues/27842

botelastic[bot] commented 1 year ago

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

mhert commented 1 year ago

I just ran into this issue this weekend, and it took some time, until I found out that I had to remove the trailing slash.

I'm pretty sure I'm not the first one, who ran into this, so I would appreciate it if this could get some priority.

botelastic[bot] commented 9 months ago

Hi! We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!