minor suggestion for code summary status report

[Adamszk]

The code summary shows three columns of code size and speed between x86, ARM and Epiphany. A little more detail on ID of CPU would be helpful. 1) are all codes compared on same CPU's (x86 for all codes are the same?) 2) X86 (dual, quad, clock etc) a name will do 3) ARM - the same as above 4) Epiphany 16-core or or 64-core. Is this summary made automatically or by contributor?

[olajep]

Agree.

Answer to questions: 1) Yes 2) Intel(R) Core(TM) i7-2760QM 3) ARMv7 Processor rev 0 (v7l) Cortex A9 @667 MHz (Parallella) 4) 16-core Epiphany-III (Parallella)

Summary is generated w/ scripts here: https://github.com/parallella/pal/blob/master/tools/regression/ No documentation ATM

Also see #203 "benchmarks: No way to get reproducible results"

[Adamszk]

@olajep I left a comment for #203 Regarding the summary, it does not look encouraging for epiphany chip. Calculations timing is supposed to be faster than standard cpu's. The results are not presented on fair footing. Here is what I did to normalize it: x86 - 4 cores at 2.400 GHz ARM - 2 cores at 667.0 MHz Epiphany - 16 cores at 600,0 MHz Modification of time results -> performance rating = (time results) * (# of cores) * (frequency) The lower number the better. Result is dimensionless. sample -> p_abs(); -> { 2.06ns, 7.88ns, 271.25ns } performance rating -> {19.78, 10.67, 2604} Observation: The first two rating makes sense. Since ARM cpu is slower by architecture (not by design), slower in frequency and has less cores, code timing should take longer which reflects that on our results. But it does not mean that it is under performer. Thus, performance rating is used to correct that to ensure fair footing comparison by normalizing hardware rating and therefore it shows that ARM is still better than x86. However, Epiphany timing is not encouraging. Either timing of Epiphany is not done properly or my formula is not correct, or my formula is not applicable to Epiphany or the programming (internally or externally (us) ) is not done correctly or we are missing something here. It shows that something is not right with Epiphany. Question: x86 and ARM is giving you result once - timing once. is Epiphany giving results 16x times - timing once? Note: the summary does not measure performance of the PAL developed code, it measures the cpu performance. To measure the PAL's code performance, it has to be compared to existing math.h library , for example, comparing absf(x) to p_abs_f32(x) -> all to give one result or 16x and both need to be tested on same CPU's which is epiphany. Therefore, we need two columns not three. Since we are code developers, we are not interesting in some x86, ARM, Epiphany hardware performance. We want to know how our code compares with standard code. Is the code better or worst than standard code? etc.... Hope that helps....