search

vivekaxl / Bellwether-Config

BEETLE: Finding New Configurations by Borrowing from Old Ones
1 stars 1 forks source link

RQ3: How to find Bellwether? #5

Closed vivekaxl closed 6 years ago

vivekaxl commented 6 years ago

SQLite -> does not show good results. We might want to change the way we select SQLite.

@rahlk Any ideas? All I can think of is a brute force?

rahlk commented 6 years ago

Which one's the bellwether?

rahlk commented 6 years ago

Btw, in spear, how can rank diff be 0.1 or 0.01?

vivekaxl commented 6 years ago

Bellwethers:

  1. sac: sac_4 -> clearly visible
  2. sqlite: sqlite_88
  3. spear: spear_7 ->clearly visible
  4. x264: x264_9 -> There can be more than one bell wether

image

vivekaxl commented 6 years ago

@rahlk

Btw, in spear, how can rank diff be 0.1 or 0.01?

Good catch -> the actual number is 0 ->I changed it to 0.01 so that it shows up on a log scale.

rahlk commented 6 years ago

But there is still a point in SPEAR less than 1, that can't be right?

rahlk commented 6 years ago

As for SQL, can we pick a subset of the datasets. Not randomly, but by doing a quick experiment? We could look for projects what show a correlation between dependent variables.

vivekaxl commented 6 years ago

But there is still a point in SPEAR less than 1, that can't be right?

why is it?

rahlk commented 6 years ago

why is it?

Well ranks are integers, so are their differences, no?

vivekaxl commented 6 years ago

Well ranks are integers, so are their differences, no?

but that rank difference can be 0

vivekaxl commented 6 years ago

Results:

sqlite : 3.682 +- 4.647 spear : 0.099 +- 0.0 sac : 0.639 +- 27.884 storm-obj2 : 0.0 +- 0.0 x264 : 1.511 +- 8.182

Doesn't work:

storm-obj1 : 2.268 +- 48.616

Notes: