microsoft / lst-bench

LST-Bench is a framework that allows users to run benchmarks specifically designed for evaluating Log-Structured Tables (LSTs) such as Delta Lake, Apache Hudi, and Apache Iceberg.
Apache License 2.0
69 stars 35 forks source link

Introduce concurrent execution of tasks within a session #337

Closed jcamachor closed 1 month ago

jcamachor commented 1 month ago

This PR introduces the ability to execute tasks concurrently within a session, providing more flexibility in workload management. For concurrent execution, each task must specify a start time relative to the beginning of the session, and the maximum number of tasks that can run in parallel should be defined at the session level. The following snippet shows the definition of a session executing tasks concurrently:

  - tasks:
    - template_id: single_user_simple
      permute_order: true
      start: 0
    - template_id: single_user_simple
      permute_order: true
      start: 10000
    - template_id: single_user_simple
      permute_order: true
      start: 20000     
    max_concurrency: 2