RQ3: How to find Bellwether?

vivekaxl / Bellwether-Config

BEETLE: Finding New Configurations by Borrowing from Old Ones

1 stars 1 forks source link

RQ3: How to find Bellwether? #17

Open vivekaxl opened 6 years ago

vivekaxl commented 6 years ago

Results:

sqlite : 3.682 +- 4.647 spear : 0.099 +- 0.0 sac : 0.639 +- 27.884 storm-obj2 : 0.0 +- 0.0 x264 : 1.511 +- 8.182

Doesn't work:

storm-obj1 : 2.268 +- 48.616

Notes:

The rank predicted by the bellwether found by the bellwether method in x264 is within the top 1.51% of the dataset.
This can also be interpreted as the distance to heaven, where the distance is normalized to the range of the dependent values.

pooyanjamshidi commented 6 years ago

One thing that we need to consider is the variance in the measurements in datasets, basically the distance between the best and worst configurations. The distance of worst configuration to heaven from one dataset vary to another depending on the variance, right?

vivekaxl commented 6 years ago

Correct. I have normalized the dependent values of all the datasets.