search

rchain / bounties

RChain Bounty Program
MIT License
90 stars 62 forks source link

how many apprentices, journeyers, and masters should there be? #785

Open dckc opened 6 years ago

dckc commented 6 years ago

Benefit to RChain

A balance in the bounty system where people doing good work are trusted to do so but misbehavior is managed effectively.

Description

In the 201805 pay period, people that had been trusted voters were losing their trust ratings as more certifications came in, and not because they were misbehaving. This is because the trust metric algorithm includes a parameter for the expected number of trusted voters, which was set at 45. So as people slipped below the top 45, they lost their trust rating.

Some suggested removing the limit on the number of trusted voters, but without some limit, the algorithm loses its attack resistance property.

On Jun 7 (https://github.com/rchain/bounties/issues/375#issuecomment-395478315), we tried raising the limit to 100, but we (@Jake-Gillberg , @jimscarver and @dckc) noticed some trusted votes we were not comfortable with, so we reverted that change.

Note the July 28 change to 60,30,20 for the 201807 pay period below.

cc @jimscarver @lapin7

Budget and Objective

suggested budget:

measure of completion:

jimscarver commented 6 years ago

It might be good to see the list of who gets in if we increase the number of apprentices to 55. As we scale the number need to increase. We may want to also bump up the number of journeyers and masters to see how that changes who gets in.

dckc commented 6 years ago

_oops; trust_view doesn't re-calculate the flow._

In case you'd rather not wait 'till I get around to it, you should be able to see who gets in by running:

python social_coding_sync.py trust_view --good_nodes=55,25,15

ref https://github.com/dckc/rchain-dbr/blob/master/social_coding_sync.py

dckc commented 6 years ago

idea from discussion with @jimscarver : record current status - ratings and what's funded bump masters and journeys by 1, apprentices by 5 look at the impact, and if it's not bad, go with it.

What do you think, @deannald ? @PatrickM727 ? (we can talk Fri 13 July if not before)

deannald commented 6 years ago

Let's discuss at our next meeting so I can get a better idea of your discussion with Jim and the possible ramifications of what you're proposing.

allancto commented 6 years ago

@dckc @jimscarver @David405 it appears that adding a new certification can affect certification of others (happened to @David405 today). This seems like unexpected behavior in two regards, first of all there is no warning about who is going to fall off as the result of a new vote and second the one who falls off is chosen by a LIFO algorithm which is not transparent.

I'd recommend a different process, if necessary a manual process where certification votes aren't recorded until approved by an authority (TAC?), or a system where there is a pool of vacancies and new additions are not allowed when count of vacancies = 0. @deannald did this get discussed already? apparently the ramifications are non trivial. Thanks! -@allancto

dckc commented 6 years ago

As to notice and transparency: that's perhaps feasible. Right now, the rewards site doesn't send mail nor write to github, but perhaps it could. There's a time when it has the new ratings in memory and the old ones in the DB and it could compare and, say, write a comment to a github issue for transparency (and github could handle notification from there).

The algorithm isn't LIFO. It's a capacity constrained flow network, as noted in Trust Ratings. Note that "falling off the bottom" isn't the only way to lose your rating. Someone can un-certify you or someone upstream from you.

What would it mean to approve a certification vote? Your vote is your vote, no?

A system with a pool of vacancies sounds like replacing the Advogato trust metric with something else. I would need to know a lot more about the something else to know whether it's an improvement.

dckc commented 6 years ago

I just deployed:

I've been trying since Jun 30 to get agreement with @deannald and @PatrickM727 but we're struggling with communications / timing / logistics.

So since this is getting in thew way of people doing good work, I'm going ahead with what feedback I've got.

deannald commented 6 years ago

Sounds good. I'm hoping to meet up with @PatrickM727 sometime in the next two weeks to discuss the Bounty System. We should follow up with a team call to get everyone on the same page.

dckc commented 6 years ago

@deannald @PatrickM727 please consider:

In #847 @allancto writes:

@dckc this unintended change gave all participants (even with weight 0) the ability to produce a quorum though not influence the amount. Perhaps that's a reasonable amount of trust to extend. (more conservative might be to require >=3 voters && a minimum total trust rating), following in the spirit of opinions expressed by @David405 and @jimscarver .

I'm still mulling it over.

dckc commented 6 years ago

In order to recover the state of May rewards for #799, I reverted the July 27 "at least three" change today.

I still have not made up my mind about how it should work going forward.

deannald commented 6 years ago

I would err on the side of using at least three trusted members - it feels like there's more checks and balances with that method.

dckc commented 6 years ago

Toward clearer trust net viz

I thought showing flow numbers on the arrows in the trust metric visualization would make it more clear why ratings end up the way they do, but the flow numbers are computed separately for each level. So I'm preparing to show a separate network for each level.

I did the back-end logic (in python); the front end (html/js) side is still todo.

https://github.com/dckc/rchain-dbr/blob/viz_flow/social_coding_sync.py a66dcb6

dckc commented 6 years ago

Oops! I just realized I hadn't updated the trust seed!

# git diff
diff --git a/social_coding_sync.py b/social_coding_sync.py
index 0db8fd3..2b5d57d 100644
--- a/social_coding_sync.py
+++ b/social_coding_sync.py
@@ -17,7 +17,7 @@ Options:
                     and _databse.db_url
                     [default: conf.ini]
   --seed=NAMES      login names (comma separated) of trust seed
-                    [default: dckc,Jake-Gillberg,kitblake]
+                    [default: dckc,deannald,PatrickM727]
   --good_nodes=XS   qty of good nodes for each rating (comma separated)
                     [default: 60,30,20]
   --view=FILE       filename for social network visualization

changes to trust ratings are in https://gist.github.com/dckc/dab60698c215bdff16008c1e150f48ac

dckc commented 6 years ago

Toward clearer trust net viz

I did the back-end logic (in python); the front end (html/js) side is still todo.

Got that part coded up now:

It's deployed to staging: http://rewards-test.rhobot.net/trust_sync/trust_net_viz.html

dckc commented 6 years ago

Relax constraints on reward votes?

How about for reward votes, input from all verified coop members counts evenly, and only 2 votes are needed for a quorum?

I have tentatively deployed (but not committed / pushed) this change:

connolly@jambox:~/projects/rchain-dbr$ git diff
diff --git a/dbr_schema/dbr_views.sql b/dbr_schema/dbr_views.sql
index 40fba5a..77b7230 100644
--- a/dbr_schema/dbr_views.sql
+++ b/dbr_schema/dbr_views.sql
@@ -107,7 +107,7 @@ select issue_num, title
         where sj.pay_period = ea.pay_period
         and sj.worker = ea.worker
        ) then 0
-       when voter_qty >= 3 then reward_usd_1
+       when voter_qty >= 2 then reward_usd_1
        else null
        end reward_usd
      , percent_avg
@@ -119,23 +119,21 @@ select issue_num, title
 from (
        select ib.pay_period, ib.issue_num, ib.title, ib.labels, rv.worker
             , count(distinct verified_coop) voter_qty
-            , group_concat(sig separator ', ') voters
+            , group_concat(rv.voter separator ', ') voters
              , ib.budget_provisional
              , ib.budget_usd
-            , round(sum(rv.percent * weight) / sum(weight), 2) percent_avg
-            , round(sum(rv.percent * weight) / sum(weight) / 100 * ib.budget_provisional) reward_provisional
-            , round(sum(rv.percent * weight) / sum(weight) / 100 * ib.budget_usd) reward_usd_1
+            , round(avg(rv.percent), 2) percent_avg
+            , round(avg(rv.percent) / 100 * ib.budget_provisional) reward_provisional
+            , round(avg(rv.percent) / 100 * ib.budget_usd) reward_usd_1
        from issue_budget_wt ib
        join (
-         select coalesce(rv.weight, uf.weight) weight
-              , concat(rv.voter, '*', coalesce(rv.weight, uf.weight)) sig
-              , uf.verified_coop
+         select rv.voter, uf.verified_coop
               , rv.issue_num, rv.pay_period, rv.percent, rv.worker
          from reward_vote rv
          join user_flair uf on uf.login = rv.voter
           where uf.verified_coop is not null
        ) rv on rv.issue_num = ib.issue_num and rv.pay_period = ib.pay_period
-       group by ib.pay_period, ib.issue_num, ib.title, rv.worker
+       group by ib.pay_period, ib.issue_num, ib.title, ib.labels, rv.worker
 ) ea
 ;
kitblake commented 6 years ago

I'm for relaxing constraints on reward votes, but I'd suggest keeping the quorum of needed votes at three. Simply for consistency.

How many votes are needed? Three. No need to explain and document that it's three for budgets and something different for rewards.

Also, with the relaxed constraint of any verified Coop member, it wouldn't hurt to have three confirming members.