Loadgen Python API does not expose server_num_issue_query_threads

mlcommons / inference

Reference implementations of MLPerf™ inference benchmarks

https://mlcommons.org/en/groups/inference

Apache License 2.0

1.22k stars 532 forks source link

Loadgen Python API does not expose server_num_issue_query_threads #1835

Open ever-wong opened 2 months ago

ever-wong commented 2 months ago

Loadgen Python API does not expose server_num_issue_query_threads in mlperf::TestSettings. It seems users cannot set this attribute directly from Python.

Will create a PR to add this in pybind module.

arjunsuresh commented 2 months ago

@ever-wong Just to understand, is this parameter affecting performance for some systems?