elastic / rally

Macrobenchmarking framework for Elasticsearch
Apache License 2.0
1.95k stars 313 forks source link

Compare does not work with tracks with "metric" index names #1802

Closed gareth-ellis closed 10 months ago

gareth-ellis commented 10 months ago

In for example the elastic/logs track we produce metric names with the index in.

E.g .ds-logs-endpoint.events.file-default-2023.11.02-000001 process.name.text norms

This means that if we want to compare two runs that were run on different days, we cannot as they produce different

Doing so produces: [ERROR] Cannot compare. '.ds-logs-endpoint.events.file-default-2023.11.01-000001'.

We should skip uncomparable metrics without erroring out.