search

HPCE / hpce-2016-cw5

0 stars 2 forks source link

Graph Times #24

Open pm1113 opened 7 years ago

pm1113 commented 7 years ago

Hello,

I was wondering how you are timing the code.

I am consistency getting times that are about 100-1000 times greater than what my individual graph shows me.

Best, Pranav

Szypicyn commented 7 years ago

To further expand on the above, In the latest series of tests (22/11/16) the reference timings are inconsistent with those obtained by myself. I have seen a case where code hasn't been touched at all, and the graph has shown a substantial speed-up

jeremych1000 commented 7 years ago

Also @Szypicyn did your last test produce graphs? I only have the tarball pushed to my repo, but no graphs.

m8pple commented 7 years ago

See #25 about the lack of graphs - there should be new ones based on a fresh run in a couple of hours.

Regarding the timing, pay careful attention to exactly what the spec says that the execution time metric is.

I'm (hopefully) measuring that metric - are you sure that is what you are measuring, or is there other stuff getting included as well?

jeremych1000 commented 7 years ago

@m8pple I'm measuring puzzler::Puzzle::Execute by extracting the times from the log, but the times are still nowhere near 10^-5s as in your graphs.

Would it be possible to add in a reference line just so we can know what the speed-up is compared to the reference?

pm1113 commented 7 years ago

I used the logs too and the times match the times show in the excel sheet. However the times dont match the times shown in the graphs. I think the axis of the graphs is off. @m8pple, @jeremych1000 could that be the case?

jeremych1000 commented 7 years ago

@pm1113 I think that is the case too. If you look in all.log in the tarballs,

Time budget used = 10.295967, remaining = 19.704033 Time budget used = 10.953205, remaining = 19.046795 Time budget used = 11.677849, remaining = 18.322151

Doesn't match too.

m8pple commented 7 years ago

Sorry, hadn't realised this was still active.

I mis-understood the question, I thought this was between people seeing their results including IO/input generation versus mine that don't (usually the problem).

I'd only looked at the graphs as a relative measure of performance, so I didn't realise it was a question about absolute times. I now see you were meaning the scale, not the actual times.

I think it is simply the scale of the graph is off by 1000, so probably I'm converting from ns to s incorrectly when plotting the actual data. The results in the csv look correct though, and if I plot them I get exactly the same shape as shown in the graphs.

qc316 commented 7 years ago

Hello,

@m8pple, Is it possible to push graphs for the final implementations ? We'd like to see how much what we have done today is useful. Thanks