ai-se / BUBBLE_TSE

BUBBLE for TSE submission
0 stars 2 forks source link

Reviewer 1,Q6: Questions regarding analysis of results #7

Open Suvodeep90 opened 4 years ago

Suvodeep90 commented 4 years ago

The analysis of the results should be substantially improved. Let consider the case of Section 5 - RQ1. The results are reported in less than half column and refer to a figure (Figure 6) which does not explain anything. The summary reports that "Theoretically and empirically, the hierarchical reason- ing on GENERAL performs much faster than standard bellwether methods": I cannot understand where these "theoretically and empirically" come from, since there is no proof of them. The same is true for the other research questions. And, in general, there is qualitative analysis: Why are the results the ones reported in the paper? What makes the reported method scalable?

1) "Theoretically and empirically, the hierarchical reasoning on GENERAL performs much faster than standard bellwether methods": I cannot understand where these "theoretically and empirically" come from since there is no proof of them.
2)  In general, there is qualitative analysis: Why are the results the ones reported in the paper? 
3) What makes the reported method scalable? 
Suvodeep90 commented 4 years ago

"Theoretically and empirically, the hierarchical reasoning on GENERAL performs much faster than standard bellwether methods": I cannot understand where these "theoretically and empirically" come from since there is no proof of them.

Theoretically - we need to include an analysis of the algorithm to prove big-O notation. Empirically - the empirical result can be seen in RQ1, which shows the algorithm/framework works much faster than the normal Bellwether method.

Suvodeep90 commented 4 years ago

What makes the reported method scalable? Theoretically and empirically the algorithm/framework is showing it is scalable. This is because as the number of clusters at the leaf node grows the algorithm/framework will be faster.

More details needed