Potential bug in ratio constraints calculation

joelkuiper commented 9 years ago

"Tijdens het uitwerken van de regorafenib case study voor het educational material ben ik tegen een bug aangelopen met betrekking het het berekenen van de ratio constrainst op basis van een non-linear partial value function. Zie hiervoor de bijgevoegde drie screenshots. Zoals je ziet op de screenshot exactTradeoff heb ik aangegeven indifferent te zijn tussen 4 months OS (criterion 1); 0% haemorrage (criterion 4) en 5 months survival; 5% haemorrage. Voor de onderliggende value function moet dus gelden dat:

w{1}v{1}(4) + w{4}v{4}(0) = w{1}v{1}(5) + w{4}v{4}(5).

Na het invullen van de waarden uit de partial value functions (screenshot PVs) wordt dit:

w_{1}0 + w{4}1 = w{1}0.5 + w{4}_0

ofwel

w{1}/w{4} = 1/0.5 = 2.

In het preference statements overview staat echter w{1}/w{4} = 3.333 (screenshot preferences). Dit lijkt dus niet te kloppen (tenzij ik zelf een denkfout gemaakt heb in bovenstaande). Het bepalen van deze constaints gaat wel goed voor de linear partial value functions."

preferences exacttradeoff pvs

gertvv commented 9 years ago

Yeah looks like Douwe is right. I don't immediately see what's wrong though:

function getRatio(state) {
  var u = criteria[state.criterionA].pvf.map;
  return 1 / u(state.choice);
}

That looks correct: 1 / u(state.choice) = 1 / v_{1}(5) = 1 / 0.5 = 2, but obviously that's not what it evaluates to. Could the criteria have been swapped?

2015-03-31 8:51 GMT-04:00 Joël Kuiper notifications@github.com:

"Tijdens het uitwerken van de regorafenib case study voor het educational material ben ik tegen een bug aangelopen met betrekking het het berekenen van de ratio constrainst op basis van een non-linear partial value function. Zie hiervoor de bijgevoegde drie screenshots. Zoals je ziet op de screenshot exactTradeoff heb ik aangegeven indifferent te zijn tussen 4 months OS (criterion 1); 0% haemorrage (criterion 4) en 5 months survival; 5% haemorrage. Voor de onderliggende value function moet dus gelden dat:

w{1}v{1}(4) + w{4}v{4}(0) = w{1}v{1}(5) + w{4}v{4}(5).

Na het invullen van de waarden uit de partial value functions (screenshot PVs) wordt dit:

w_{1}0 + w{4}1 = w{1}0.5 + w{4}_0

ofwel

w{1}/w{4} = 1/0.5 = 2.

In het preference statements overview staat echter w{1}/w{4} = 3.333 (screenshot preferences). Dit lijkt dus niet te kloppen (tenzij ik zelf een denkfout gemaakt heb in bovenstaande). Het bepalen van deze constaints gaat wel goed voor de linear partial value functions."

[image: preferences] https://cloud.githubusercontent.com/assets/971293/6919199/461e73d2-d7b5-11e4-97ce-fa40823e71d5.png [image: exacttradeoff] https://cloud.githubusercontent.com/assets/971293/6919201/4b1d63e8-d7b5-11e4-8a7f-138e04aa65ae.png [image: pvs] https://cloud.githubusercontent.com/assets/971293/6919205/4ffbdfb6-d7b5-11e4-823a-cd55f7f17252.png

— Reply to this email directly or view it on GitHub https://github.com/drugis/mcda-elicitation-web/issues/85.

joelkuiper commented 9 years ago

The correct answer is "I don't know", but seems unlikely … I'll take a look, but it seems a hard one to tackle!

joelkuiper commented 9 years ago

So it seems I fixed this by sorting the cutoffs when the PVF is increasing. But maybe you can take a look as well (time permitting of course)?

gertvv commented 9 years ago

I'm not 100% certain this is related, but the scales on the PVF creation sliders now get messed up. The first step is also a bit weird. Take "overall survival" in the regoranefib dataset.

By default it is set to increasing. However, when I start eliciting it, the default is set to "4 months is best" instead of "8 months is best". I set it to 8 months.
In the first step I choose "5 months". In the second step I choose "4.5 months". Then, in the third step, the slider runs from "4.5 months" to "8 months". Obviously that should be "5 months" to "8 months".

Edit: also interesting, the latter bug does not happen when eliciting a decreasing piece-wise linear function.

gertvv commented 9 years ago

Looks like the original bug is fixed though.

gertvv commented 9 years ago

No, it isn't. For example, if I say "4 months survival, 0% haemorrhage" = "4.5 months survival, 5% haemorrhage", I would expect that 0w_1 + 1w_4 = 0.25w_1 + 0w_4 w_1/w_4 = 4 But the user interface tells me that it is 1.38 (or something). Conversely, when I select 6.5 months (which should have value 0.75), I expect that w_1/w_4 = 1 1/3, but it is not (it's around 4).

gertvv commented 9 years ago

Of note, if I make the PVF linearly increasing, the results are correct with the following cut-offs: "6 months" => w_1/w_4 = 2 "5 months" => w_1/w_4 = 4 "7 months" => w_1/w_4 = 1 1/3 The same for linearly decreasing, I get: "6 months" => w_1/w_4 = 2 "5 months" => w_1/w_4 = 1 1/3 "7 months" => w_1/w_4 = 4

So I suspect that the inverse of piece-wise linear functions is not being computed correctly somehow.

gertvv commented 9 years ago

Now a decreasing piece-wise linear function, with the first cut-off at 7, and the second and third at the mid-points. I get: "7 months" => w_1/w_4 = 2 "5.5 months" => w_1/w_4 = 1 1/3 "7.5 months" => w_1/w_4 = 4 Which is correct. So it is only going wrong when the function is piece-wise linear AND increasing.

drugis / mcda-elicitation-web

Potential bug in ratio constraints calculation #85