search

elastic / elasticsearch

Free and Open, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.53k stars 24.61k forks source link

Re-enable usage of `copy_to` in `security-metricbeat.json` #112450

Open salvatore-campagna opened 1 week ago

salvatore-campagna commented 1 week ago

Description

With PR https://github.com/elastic/rally-tracks/pull/652 we had to disable usage of copy_to in elastic/security since Synthetic Source still lacks support for it. As a workaround we modified the Rally elastic/security track to conditionally use copy_to if index_mode is not logsdb.

After merging a PR fixing https://github.com/elastic/elasticsearch/issues/110038 and supporting copy_to in Synthetic Source, we need to revert PR https://github.com/elastic/rally-tracks/pull/652 making sure that queries run targeting the message field instead of kubernetes.event.message.

This will also allow us to double check that copy_to works as expected, after making sure the track runs without issues in nightly benchmarks. After doing this take the chance to evaluate the performance impact of using copy_to with Synthetic Source. We should expect some performance penalty here end enabling usage of copy_to after supporting it will give us a chance to do so in Rally nightly benchmarks without running any benchmark manually.

elasticsearchmachine commented 1 week ago

Pinging @elastic/es-storage-engine (Team:StorageEngine)