Preset Scalable Likert Reward Curves

[elisemarika]

Job to be Done When I'm deciding whether to participate in a Marketplace, I want to understand its reward curve, so I can ensure I'm comfortable with the way rewards are distributed before I commit to doing work and spend time, that way I maximize my limited bandwidth.

Requirements Update screens

• Update Create Marketplace form >> reward curve dropdown
• Update Marketplace / Rewards >> reward curve block
• Update Launch Challenge form >> reward curve dropdown
• Update Challenge / Rewards >> rewards section explainer copy

Support all enforcement criteria

• Scalable Likert with various distro buckets
• https://docs.google.com/spreadsheets/d/1rdzJCO_cuuPcE-I0aEorrcj5fiBWkT3YAgQeqYlcEeI/edit?usp=sharing

Design Dev Ready: https://www.figma.com/file/TVCoREiQSgnWg2J5qrHKP0/MDAO-App-v2?type=design&node-id=34-7235

[elisemarika]

Drake idea -- can we display a chart example so people can visualize how it works

[drakedanner]

Right now, we're using a curve we call "Constant Likert"

We have a method to introduce a parameterized curve called "Scalable Likert"

Within the Scalabale Likert world, GJ has helped us come up with "Aggressive", "Acceptable", and "Pass/Fail"

You can duplicate this sheet to try out some data entry and see how things would play out

You can also use or duplicate the Test Here sheet to try to design a new curve within the Scalable Likert format

https://docs.google.com/spreadsheets/d/1UoIyaMX-_AoSIQC8T_wDDIBoxug9nyCyJQvBNfXqG6c/edit?usp=sharing

The Default tab is what "Constant Likert" looks like

You can enter values in column B (the total cumulative score) and c (the number of scores) to arrive at a avg score

Then you can see how the payouts are calculated

On the scalable curve, you can enter values in columns N (the bucket weight) and O (the bucket range) to apply greater weighting to different score distributions, this is the "Scalable" piece

for example, you can see that the "Aggressive" curve only rewards scores above 74 and it rewards scores above 89 with greater value than scores 75-89

We can follow this structure to produce a variety of curves, so if you come up with one that you think makes sense please share it!

We're thinking through how to expose this in the UI too. With the understanding that having people enter values for bucket ranges and distributions may be quite complicated, we'd like to have a set of pre-defined curves....

• Constant Likert - reward all submissions, according to score
• Aggressive - reward top submissions, accrording to score
• Acceptable - reward acceptable submissions, according to score
• Pass/Fail - reward all Pass scores equally

[drakedanner]

NOTE: COMMENT ABOVE IS NO LONGER ACCURATE BUT I AM LEAVING IT THERE

This is the most up to date spreadsheet: https://docs.google.com/spreadsheets/d/1rdzJCO_cuuPcE-I0aEorrcj5fiBWkT3YAgQeqYlcEeI/edit?usp=sharing

Copy/Pasting explanation from https://github.com/MetricsDAO/xyz/issues/661#issuecomment-1535360991

Working on v2 enforcement and payments and have come across the reality that we cannot support unlimited Submissions or unlimited Reviews.

The limit value for both of these will inform the maximum amount of payout possible per action.

This also empowers the operators to have greatest control over budget.

They can determine how much each Review will be paid, if less than the limit is reached then the remaining funds can be claimed back by the Sponsor.

In the case of Provider payments, each user will have a potential share of the total pool based on the limits. ie: $1000 in a Challenge with a limit of 10 means that each Provider can earn up to $100. If single user submits to the challenge and scores a 7/10 they may earn $70. Again, the remaining $30 + $900 can be reclaimed by the Sponsor. The amount of payment they receive relative to score can be scaled up and down... scalable likert!

In the case of the limit being met (10 users show up), the experience is similar to the Scalable Likert example we've used with the pool + a remainder that goes back to the Sponsor.

Since the pool is cut into potential earning slots by the limit (ie: $1000 with limit of 10, each slot is worth $100)

Bucket Ranges define a range that the score can fall within. Bucket Distribution defines the % of the slot that the participant earns if they fall into a certain range.

Some examples of potential outcomes: