The benchmarking suite, ionbench, currently assumes the implementation under test is fully functional. Some checks should be added to assert the return code from the methods tested (such as dictionary_create, dictionary_insert, etc) are returning err_ok, and not something bad like err_out_of_memory.
Possible talking points:
These checks will cause a small amount of delay, resulting in slightly higher timing measurements. Is this acceptable? Do we adjust for this in any way?
These checks could be made redundant if we were to deploy solid unit tests on device. Then the behavior of the implementation can be verified before we benchmark.
The benchmarking suite,
ionbench
, currently assumes the implementation under test is fully functional. Some checks should be added to assert the return code from the methods tested (such asdictionary_create
,dictionary_insert
, etc) are returningerr_ok
, and not something bad likeerr_out_of_memory
.Possible talking points: