search

microsoft / ADBench

Benchmarking various AD tools.
MIT License
107 stars 40 forks source link

Guaranteed timeout checking #159

Closed mikhailnikolaev closed 5 years ago

mikhailnikolaev commented 5 years ago

Global runner script checks guaranteed timeouts now. That means that tests of bigger sizes won't be run if tests of smaller sizes are interrupted due to timeout.

tomjaguarpaw commented 5 years ago

Ideally I would like to find a different word than "certain". It's a bit ambiguous since it can also mean something more like "particular", as in "certain people have been working on the ADBench project". I don't have a particularly compelling suggestion but "sure" and "guaranteed" are probably OK.

mikhailnikolaev commented 5 years ago

This looks like it will be very useful! Do we know how much this speeds up a full run?

Actually, we don't know yet, because full run is performed once a week. So, I hope this PR will finish this week till friday, thus the new full run will use a new version of the global runner.

mikhailnikolaev commented 5 years ago

Ideally I would like to find a different word than "certain". It's a bit ambiguous since it can also mean something more like "particular", as in "certain people have been working on the ADBench project". I don't have a particularly compelling suggestion but "sure" and "guaranteed" are probably OK.

Certain timeout now is called guaranteed timeout