search

ibmcb / cbtool

Cloud Rapid Experimentation and Analysis Toolkit
Apache License 2.0
78 stars 49 forks source link

RUN_LIMIT doesn't work well for many workloads #446

Open rayx opened 3 months ago

rayx commented 3 months ago

Take nullworkload's start script cb_run_nothing.sh as an example, it has the following layout:

cb_run_nothing.sh
  -> call execute_load_generator to run workload and generate result
  -> call cb_report_app_metrics.py to upload metrics

The issue is that, while execute_load_generator checks RUN_LIMIT, the code that calls cb_report_app_metrics.py doesn't. As a result, although workload is skipped in execute_load_generator, cb_report_app_metrics.py keeps posting data to metrics store. I think many workload's scripts have this issue.