Optimizations, performance, and benchmarking

[LaCuneta]

Benchmark - No Optimizations

BZ Benchmark (V8 Node.js v6.10.3):

• Average: 54.561 seconds
• Min: 45.171 seconds
• Max: 84.966 seconds

FireBig Benchmark (V8 Node.js v6.10.3):

• Average: 96.768 seconds
• Min: 76.092 seconds
• Max: 128.195 seconds

GridWalk Benchmark (V8 Node.js v6.10.3):

• Average: 446.069 seconds
• Min: 340.928 seconds
• Max: 581.184 seconds

Wolf Benchmark (V8 Node.js v6.10.3):

• Average: 336.694 seconds
• Min: 250.894 seconds
• Max: 455.119 seconds

Including All Optimizations

BZ Benchmark (V8 Node.js v6.10.3):

• Average: 38.339 seconds
• Min: 35.332 seconds
• Max: 41.527 seconds

FireBig Benchmark (V8 Node.js v6.10.3):

• Average: 48.298 seconds
• Min: 47.521 seconds
• Max: 49.094 seconds

GridWalk Benchmark (V8 Node.js v6.10.3):

• Average: 276.59 seconds
• Min: 274.283 seconds
• Max: 280.762 seconds

Wolf Benchmark (V8 Node.js v6.10.3):

• Average: 282.732 seconds
• Min: 185.238 seconds
• Max: 439.836 seconds

[SethTisue]

the spread between the min and max numbers seems alarmingly high for some of these? curious about that, wondering if those numbers are meaningful

[LaCuneta]

@SethTisue That's a good question, I'll have to look into that! It could be something as simple as processes on my workstation interfering with the results as they ran. If the random seed isn't set the same, that'd also cause big differences, but I'm pretty sure it is. I can also get SpiderMonkey setup and check if there are similarly wide gaps between min/max using it.

I did run manual stopwatch tests as well in the browser to confirm the results, and did see improvements that came close to the averages here. I did not see the big differences in min/max, though, when doing it that way.