In serverless, data is considered hot when document timestamps are within the boost window and cold when timestamps are outside the boost window. Tracks using date ranges for corpus generation are limited to static dates, presenting a problem for nightly benchmarks where all ingested data is indexed cold.
This commit adds two custom Jinja filters to the track loader to create dynamic date ranges. These simple functions are most useful in the serverless context for any track utilizing date ranges for corpus generation. Usage:
get_start_date(format)
Pair with the now global template variable to produce a start date:
"start-date": {{ now | get_start_date('%Y-%m-%d') | tojson }}
Result:
"start-date": "2024-10-22"
get_end_date(duration_days, format)
"end-date": {{ now | get_end_date(3, '%Y-%m-%d') | tojson }}
Result:
"end-date": "2024-10-25"
Documentation
Documentation will be added with a subsequent commit to this PR. The documentation on the now global filter does not mention the return value is a float as it is the result of time.time().
In serverless, data is considered hot when document timestamps are within the boost window and cold when timestamps are outside the boost window. Tracks using date ranges for corpus generation are limited to static dates, presenting a problem for nightly benchmarks where all ingested data is indexed cold.
This commit adds two custom Jinja filters to the track loader to create dynamic date ranges. These simple functions are most useful in the serverless context for any track utilizing date ranges for corpus generation. Usage:
get_start_date(format)
Pair with the
now
global template variable to produce a start date:Result:
get_end_date(duration_days, format)
Result:
Documentation
Documentation will be added with a subsequent commit to this PR. The documentation on the
now
global filter does not mention the return value is a float as it is the result oftime.time()
.