search

aws-samples / foundation-model-benchmarking-tool

Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
https://aws-samples.github.io/foundation-model-benchmarking-tool/
MIT No Attribution
182 stars 27 forks source link

update create combination with test count limit #124

Closed haozhx23 closed 3 months ago

haozhx23 commented 3 months ago

Issue #, if available: N.A. (No issue, optimizations only)

Description of changes:

  1. Limit the test counts to a range when dataset length and concurrency differs;
  2. The above range will be read from test config file in field "experiments", and will take 5 as the default value for min_iters_per_combination, 50 for max_iters_per_combination.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.