search

HPCE / hpce-2016-cw5

0 stars 2 forks source link

Evaluating julia's speedup #14

Closed jeremych1000 closed 7 years ago

jeremych1000 commented 7 years ago

How should we evaluate julia's speedup when doing CPU, OpenCL, and more optimizations? The input c is randomized every time, and sometimes it takes forever if the while loops run for a long time.

This makes it difficult to benchmark the improvements I've made - is it wrong to hardcode a value of c just to benchmark?

m8pple commented 7 years ago

The intrinsic variability of the problem is deliberate, as some workloads are highly data-dependent.

So hard-coding c would probably be a bad idea - you want to test the bad cases and the good (though particularly the bad).

Note that you have the ability to generate workloads of a fixed difficulty, as you have the ability to generate fixed inputs (create_puzzle_input) then store them as a file. That input can then be played back through many different implementations, always of the same hardness. It's also a good way of calibrating improvement over time, by capturing a selection of "golden" inputs.

jeremych1000 commented 7 years ago

Thanks, didn't look at create_puzzle_input.