Robustness (Metrics discussion)

[AngelaKTE]

Description

• Goal: We measure how sensitive a voting rule is to minimal changes in the votes
• Why here?:
  • we know that today it's hard for voters to define the exact funding a project deserves (down to the single 1 token!, impact = profit!)
  • many voters go with rounded numbers (10.000/50.000/100.000), or assign the same vote/funding to a number of project they perceive as similar, but not necessarily as equal (add data analysis prior voting rounds?)
  • a comparably "lazy" voting rule would not react much on a single vote change, hence would support this rough assignment of votes
  • a comparably "sensitive" voting rule reflects more precisely how votes are assigned, down to the tiniest change by a single voter
• How we model it:
  • we measure the distance between two voting outcomes using l1 distance (specify l1 distance, also known as Manhattan distance or taxicab distance)
  • according to robustness.ipynb, the random_change_vote function takes a vote and it modifies exactly one of the votes (randomly chosen) by assigning it a new random (floating-point) value between 0 and 1 .
• We compare:
  • the change of votes (in ell1 distance)
  • the change in the outcome (in ell1 distance)
  • across all voting rules
• Chart:
  • x-axis: change of votes (in ell1 distance)
  • y-axis: change in the outcome (in ell1 distance)

Open questions:

• changing the vote between 0 and 1 is not a realistic scenario, most voters assign full token numbers (not sure if they can vote with floating numbers!), still a valid way to model to reflect "minor changes"?
• show relation between change and outcome in chart! (linear? exponential?)

[linear[bot]]

GOV-24 Robustness (Metrics discussion)

#### Description * **Goal:** We measure how sensitive a voting rule is to minimal changes in the votes * **Why here?:** * we know that today it's hard for voters to define the exact funding a project deserves (down to the single 1 token!, impact = profit!) * many voters go with round numbers, or assign the same vote/funding to a number of project they perceive as similar * a comparably "lazy" voting rule would not react much on a single vote change, hence would support this rough assignment of votes * a comparably "sensitive" voting rules reflects more precisely how votes are assigned, down to the tiniest differences by a single voter * **How we model it:** * * **We compare:** * we compare the voting rules * in the simulation, * in the simulation, we randomly select the target project who should receive the `desired_increase` **Chart:** * we plot the cost of bribery per desired_increase for all voting rules evaluated (not average across all desired_increase) * x-axis: `desired_increase` * y-axis: cost of bribery **Open questions:** the code seems not final? `def simulate_bribery_median(model, target_project, desired_increase): ` `…` `bribery_cost = desired_increase` ?

[AngelaKTE]

@nimrodtalmon77 Could you check the Open questions above and make a comment?

Copying Nimrod's answer from Linear here:

Hmm.. regadrding the open questions:

(1) I'd say that what's interesting is the dependency between how much we change a vote and how much the outcome changes. For this, a plot where the x-axis is the change in the vote (this could be measured by ell1) and the y-axis is the change in the outcome of the election (this could also be measured by ell1) is natural.

(2) I'm not sure I fully get this, but I think that the ordering doesn't matter really

[AngelaKTE]

Thanks @nimrodgithub134 !

Exactly, we'd like to understand how the change in the voting affects the outcome. I've updated the chart description accordingly.

On ordering: What I'd like to understand by this metric is a) how much the outcome changes based on vote changes (see above) b) how changes and outcome are related. The charts plotted in https://github.com/GovXS/OP-Evaluating-Voting-Design-Tradeoffs-for-Retro-Funding-RESEARCH-/blob/main/evaluations/robustness.ipynb (x-axis: rounds) show random changes, as the changes in rounds are. That's why I suggested ordering of changes, to gain a picture if the outcome ell1 and change ell1 change e.g. linearly. Makes sense?