bigfatnoob
commented
8 years ago In the first set of plots (root cost, root benefit, etc.), what is the x-axis? Is this the number of decisions/options that we have fixed (i.e. "X" in Step 5 of Slide 3 in your presentation)?
Yes. Its the number of decisions in the order as shown in the table below the plot. URL
I'm still not understanding the very strong inverse correlation between median and IQR on all of these plots. I understand that, as we fix more options, there is less to vary, and so the IQR should decrease, but the relationship seems too strong.
As we can see from the table, the most important decisions are not to set the leaves in the right subtree(Ranks 1 and 2). So once these decisions are set/unset, we only play around the left subtree and for valid solutions in the left subtree have a small range for each objective(Root cost varying between 23-28 and root benefit between 33-36) thus the small IQR. @timm could give his insights on it
Is this an artifact of our model (small number of options, so the cost and benefit distributions are highly quantized)? Or is it an artifact of the hard floor and ceiling on the cost and benefit - as the median approaches the floor or ceiling value, the lower or upper quartile shrinks and distorts the IQR metric? IQR is used to minimize the effect of outliers - do we have outliers?
Its more to do with the former as we have lesser number of costs. Hard Ceiling bound does not play a major role in this as the same pattern is observed when we do not limit the cost(as observed here).
I'm hoping for an intuitive explanation for this, Or a counter-example that shows a weaker relationship between these metrics. softgoals/weekly-reports/2016-03-04/early_termination/sample.md
I assume you are referring to the median and IQR when you say metrics. In this model we have (neil1), this is the nature of the relationship between them. For a weaker relationship I would recommend you to look at our analysis for the softgoal models(over here) where we struggle to draw a conclusion. The softgoal report for all the models we have can be found here.
@timm Ur thoughts?
sei-jklein
@bigfatnoob @timm For example, softgoals/weekly-reports/2016-03-04/early_termination/sample.md
In the first set of plots (root cost, root benefit, etc.), what is the x-axis? Is this the number of decisions/options that we have fixed (i.e. "X" in Step 5 of Slide 3 in your presentation)?
I'm still not understanding the very strong inverse correlation between median and IQR on all of these plots. I understand that, as we fix more options, there is less to vary, and so the IQR should decrease, but the relationship seems too strong.
Is this an artifact of our model (small number of options, so the cost and benefit distributions are highly quantized)? Or is it an artifact of the hard floor and ceiling on the cost and benefit - as the median approaches the floor or ceiling value, the lower or upper quartile shrinks and distorts the IQR metric? IQR is used to minimize the effect of outliers - do we have outliers?
I'm hoping for an intuitive explanation for this, Or a counter-example that shows a weaker relationship between these metrics. softgoals/weekly-reports/2016-03-04/early_termination/sample.md