search

HPCE / hpce-2017-cw5

1 stars 6 forks source link

edit_distance stops at ~5sec #58

Open adamsolomou opened 6 years ago

adamsolomou commented 6 years ago

Looking at latest performance commit edit distance stops at problem_size=8410 and accumulated time ~5sec << time budget (30sec).

csv file:

screen shot 2017-11-23 at 15 15 52

all.log :

screen shot 2017-11-23 at 15 22 05

However when I run the same code (master branch) on AWS (same instance) it runs for much greater input sizes with no issue and outputs are correct.

My test script:

screen shot 2017-11-23 at 15 24 33

My result (same for all input sizes):

screen shot 2017-11-23 at 15 23 27

Is this a problem related to my implementation or with the test scripts used by @m8pple ?

P.S: It seems that other groups have the same issue.

m8pple commented 6 years ago

One possibility is that it is related to the hacking needed to try and get around instances freezing (see #47). The logfile from the auto run suggests that a timeout is being observed at 33.91 seconds, but this is something printed out by a memory+time constraint script that I added in (not part of the pre-existing scripts). I'm somewhat suspicious of the results, but don't really have a way to check them right now.

Ultimately, if you're testing your implementation on a wide range of sizes on the AWS machine, then everything is fine. Personally I would test on a wider variety of scales than you have (your's look a bit too uniformly spaced, and are disconcertingly rounded), but your testing approach looks fine.