First, thank you for creating this useful benchmark suit.
Looking at currently posted results it's unclear if some tools actually use multiple cores. I tried to replicate these results using 24 core xeon system with 128gb running Ubuntu 20.04 and it looks like tensorflow based implementations actually run using multiple cores, but other benchmarks are only utilizing 1 core.
I assume this isn't desired effect, as all tools should probably be measured using same resources.
I've modified run-all.ps1 to limit runners to single core by changing run_command function as follows:
function run_command ($indent, $outfile, $timeout, $cmd) {
$args = "-c 0 $cmd $args"
$cmd = "/usr/bin/taskset"
write-host "Run [$cmd $args]"
Hi,
First, thank you for creating this useful benchmark suit.
Looking at currently posted results it's unclear if some tools actually use multiple cores. I tried to replicate these results using 24 core xeon system with 128gb running Ubuntu 20.04 and it looks like tensorflow based implementations actually run using multiple cores, but other benchmarks are only utilizing 1 core.
I assume this isn't desired effect, as all tools should probably be measured using same resources.
This is what I see for GMM using bench suit as is for some tools: https://matlogica.com/pub/ADBench/graphs/static/Release/jacobian/GMM%20(10k)%20%5bJacobian%5d%20-%20Release%20Graph.png
I've modified run-all.ps1 to limit runners to single core by changing run_command function as follows: function run_command ($indent, $outfile, $timeout, $cmd) { $args = "-c 0 $cmd $args" $cmd = "/usr/bin/taskset" write-host "Run [$cmd $args]"
Using single core on this system, I get the following results: https://matlogica.com/pub/ADBenchST/graphs/static/Release/jacobian/GMM%20(10k)%20%5bJacobian%5d%20-%20Release%20Graph.png
As you can see, if TF is utilizing single core, it runs more like "C++ Manual Eigen Vector" and "C++ Manual" is significantly faster.
Hope this is useful.
Kind regards, Dmitri.