search

elastic / apm-server

https://www.elastic.co/guide/en/apm/guide/current/index.html
Other
1.22k stars 519 forks source link

APM Integration Check Not Compatible With Elastic APM Data Option #13064

Open Evesy opened 5 months ago

Evesy commented 5 months ago

APM Server version (apm-server version): 8.13.2

Description of the problem including expected versus actual behavior:

When using APM server standalone (not fleet managed) along with xpack.apm_data.enabled: true to install relevant APM index templates, APM server's integration check fails due to a mismatch between the template names that are installed, and the template names APM server expects

Elasticsearch creates templates with the names, e.g. traces-apm@template, however the integration check expects the index template to be named e.g. traces-apm

Steps to reproduce:

Please include a minimal but complete recreation of the problem, including server configuration, agent(s) used, etc. The easier you make it for us to reproduce it, the more likely that somebody will take the time to look at it.

  1. Create 8.13.2 elasticsearch instance with xpack.apm_data.enabled: true
  2. Deploy 8.13.2 APM server using the above elasticsearch instance as its output

Provide logs (if relevant):

{"log.level":"error","@timestamp":"2024-05-01T10:27:05.677Z","log.logger":"beater","log.origin":{"function":"github.com/elastic/apm-server/internal/beater.waitReady","file.name":"beater/waitready.go","file.line":62},"message":"precondition 'apm integration installed' failed: error querying Elasticsearch for integration index templates: unexpected HTTP status: 404 Not Found ({\"error\":{\"root_cause\":[{\"type\":\"resource_not_found_exception\",\"reason\":\"index template matching [metrics-apm.service_summary.60m] not found\"}],\"type\":\"resource_not_found_exception\",\"reason\":\"index template matching [metrics-apm.service_summary.60m] not found\"},\"status\":404}): to remediate, please install the apm integration: https://ela.st/apm-integration-quickstart","service.name":"apm-server","ecs.version":"1.6.0"}
simitt commented 5 months ago

Thanks for reporting. The flag xpack.apm_data.enabled: true is not yet enabled by default, nor officially supported, in apm-server as there is additional work required in the server before we can switch over to using the apm plugin in Elasticsearch. We are eager to driving this work forward in the near future, but cannot yet commit to a timeline.

Evesy commented 5 months ago

Thanks for the response @simitt

I had arrived at xpack.apm_data.enabled following from https://github.com/elastic/elasticsearch/pull/97546 to allow running APM server without needing to use Fleet etc (Since in 8.x APM server does not manage its own templates)

Have I confused these two things as being the same and they are in fact solving different problems, or are they both linked and the support for it to work isn't quite there yet?

simitt commented 5 months ago

Enabling xpack.apm_data.enabled is indeed the future path for not relying on Fleet for the template + ingest pipeline setup. We just need to ensure that nothing breaks and everything is aligned and working as expected before we can switch this on by default and not rely on Fleet anymore. Not quite there yet, but it does have priority for us to drive this forward.

simitt commented 4 months ago

This should be fixed once https://github.com/elastic/apm-server/issues/11529 is done (and released).