Save interpolated query

[nineinchnick]

Saving the executed statement will make it much easier to include it in any report generated for benchmark results. It's important to include it for anyone reading the report for the first time, or generally not being familiar with the standard benchmarks, like tpch, and how its queries are different (q01 vs q09, etc.).

This assumes that the statement is identical for every execution, so it's saved as a benchmark attribute, not execution attribute.

[radek-kondziolka]

Is tag sufficient for you? If yes, concurrency-40-4-64-tracker2-oss-395-S-445cd3512d--c11

On Wed, Sep 28, 2022 at 5:05 PM Jan Waś @.***> wrote:

@.**** commented on this pull request.

In benchto-driver/src/main/java/io/trino/benchto/driver/listeners/BenchmarkServiceExecutionListener.java https://github.com/trinodb/benchto/pull/49#discussion_r982527925:

@@ -138,6 +138,9 @@ public Future<?> benchmarkFinished(BenchmarkExecutionResult benchmarkExecutionRe if (benchmarkExecutionResult.getUtcEnd() != null) { builder.withEndTime(benchmarkExecutionResult.getUtcEnd().toInstant()); }

• benchmarkExecutionResult.getExecutions().stream()

How can we confirm that? Can we find some existing throughput benchmarks?

— Reply to this email directly, view it on GitHub https://github.com/trinodb/benchto/pull/49#discussion_r982527925, or unsubscribe https://github.com/notifications/unsubscribe-auth/AWP6ELPJ6FDASRPYQKHXUBDWARNFBANCNFSM6AAAAAAQXZ3M7Q . You are receiving this because your review was requested.Message ID: @.***>

[nineinchnick]

This points to a failed tpcds benchmark, with a concurrency of 64, and 64 executions. I do see that every execution has an attribute like this:

┌─[ RECORD 1 ]───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ execution_id │ 2276695                                                                                                                                                                                                                                                                                                                                                                                                                             │
│ name         │ queries_order                                                                                                                                                                                                                                                                                                                                                                                                                       │
│ value        │ q96,q68,q79,q09,q56,q27,q23,q43,q10,q39,q13,q81,q03,q37,q42,q54,q77,q25,q17,q75,q92,q44,q65,q08,q98,q12,q97,q60,q74,q14,q61,q95,q52,q71,q57,q78,q35,q58,q02,q07,q49,q36,q18,q26,q76,q40,q80,q24,q69,q34,q47,q22,q73,q11,q30,q88,q93,q28,q31,q70,q83,q21,q55,q38,q59,q67,q06,q20,q01,q82,q99,q19,q32,q94,q50,q66,q91,q86,q48,q87,q41,q85,q89,q05,q51,q62,q45,q84,q33,q90,q63,q16,q64,q46,q72,q53,q04,q15,q29,q96,q68,q79,q09,q56,q27 │
└──────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
(1 row)

Every execution has measurements like this:

┌──────────┬────────────────────┬──────┬───────┐
│    id    │        name        │ unit │ value │
├──────────┼────────────────────┼──────┼───────┤
│ 25483178 │ queries_successful │ NONE │   102 │
│ 25483179 │ queries_failed     │ NONE │     1 │
└──────────┴────────────────────┴──────┴───────┘
(2 rows)

Thanks, I didn't know throughput benchmarks record executions in such a different way.

[nineinchnick]

@radek-starburst @przemekak PTAL again

[nineinchnick]

@sopel39 PTAL, this should be ready to be merged now (same as #51)