elastic / rally

Macrobenchmarking framework for Elasticsearch
Apache License 2.0
37 stars 314 forks source link

Add Jinja filters for creating dynamic date ranges based on now() #1884

Closed inqueue closed 3 weeks ago

inqueue commented 3 weeks ago

In serverless, data is considered hot when document timestamps are within the boost window and cold when timestamps are outside the boost window. Tracks using date ranges for corpus generation are limited to static dates, presenting a problem for nightly benchmarks where all ingested data is indexed cold.

This commit adds two custom Jinja filters to the track loader to create dynamic date ranges. These simple functions are most useful in the serverless context for any track utilizing date ranges for corpus generation. Usage:

get_start_date(format)

Pair with the now global template variable to produce a start date:

"start-date": {{ now | get_start_date('%Y-%m-%d') | tojson }}

Result:

"start-date": "2024-10-22"

get_end_date(duration_days, format)

"end-date": {{ now | get_end_date(3, '%Y-%m-%d') | tojson }}

Result:

"end-date": "2024-10-25"

Documentation

Documentation will be added with a subsequent commit to this PR. The documentation on the now global filter does not mention the return value is a float as it is the result of time.time().