Octane 2.0 includes 17 tests which are executed sequentially. With the concept of web workers evolving, the web pages give much faster response compared to the situation when all the javascript code is being made to run in one single main thread (sequential running). But Octane benchmark does not incorporate this factor into its overall scoring.
I modified the code to run each BenchmarkSuite in a different worker thread and the result was as follows:
i. The individual score decreases.
ii. The overall time taken to complete all the benchmark tests decreases.
Though the drop in the overall test completion time is quite high, this is not accounted in the final score computation, which if accounted for, could have improved the final Octane score. As a suggestion, the total time taken to complete all the tests in the benchmark could have been one of the score determining factors as a measure of better performance with parallel processing (web workers) and better cpu utilization.
I modified the code to run the suite-tests in separate threads, wherein a maximum of n threads was being allowed to run at a given point of time and then plotted the behavior as I vary this value of n on a given system.
The picture of the graph attached shows the average ‘drop in overall running time’ and ‘drop in overall octane score (in percentage), when being compared to the average time and score of the original code, with the value of n varying from least to maximum possible (1-15). It is noticeable how the overall execution time drops with the parallel running javascript code. Shouldn’t this sharp drop in the overall running time be incorporated in the determining the overall octane score?
(The experiment was performed on the chrome browser(v 43.0.2357.125) running on ubuntu 12.04LTS , of a Samsung laptop(model- NP300V5A) with processor details as- Intel Core i7-2670QM CPU @ 2.20GHz x 8 .)
Octane 2.0 includes 17 tests which are executed sequentially. With the concept of web workers evolving, the web pages give much faster response compared to the situation when all the javascript code is being made to run in one single main thread (sequential running). But Octane benchmark does not incorporate this factor into its overall scoring.
I modified the code to run each BenchmarkSuite in a different worker thread and the result was as follows: i. The individual score decreases. ii. The overall time taken to complete all the benchmark tests decreases.
Though the drop in the overall test completion time is quite high, this is not accounted in the final score computation, which if accounted for, could have improved the final Octane score. As a suggestion, the total time taken to complete all the tests in the benchmark could have been one of the score determining factors as a measure of better performance with parallel processing (web workers) and better cpu utilization.
I modified the code to run the suite-tests in separate threads, wherein a maximum of n threads was being allowed to run at a given point of time and then plotted the behavior as I vary this value of n on a given system.
The picture of the graph attached shows the average ‘drop in overall running time’ and ‘drop in overall octane score (in percentage), when being compared to the average time and score of the original code, with the value of n varying from least to maximum possible (1-15). It is noticeable how the overall execution time drops with the parallel running javascript code. Shouldn’t this sharp drop in the overall running time be incorporated in the determining the overall octane score? (The experiment was performed on the chrome browser(v 43.0.2357.125) running on ubuntu 12.04LTS , of a Samsung laptop(model- NP300V5A) with processor details as- Intel Core i7-2670QM CPU @ 2.20GHz x 8 .)