search

rchain / bounties

RChain Bounty Program
MIT License
90 stars 62 forks source link

Goals, objectives and metrics for the bounty program (reboot) #783

Open dckc opened 6 years ago

dckc commented 6 years ago

The 201805 Bounty Progress report (issue #759), had no statement of the objectives and evaluation criteria for the bounty program, which was noted as a deficiency during the board meeting Friday, June 15, 2018.

The purpose of this issue is to establish Goals, Objectives and Metrics for the bounty system.

In response to feedback after RCon3, we are "rebooting" the trust metric and label guides, effective Sep 19. To get certified to vote / guide an area:

Note earlier draft initiated by @allancto :

Benefit to RChain

A culture of return on investment in the bounty system.

Clarity for the CONTIBUTING.md, our bounty-contract.

Budget and Objective

Estimated Budget of Task: $1200 Estimated Timeline Required to Complete the Task: 6 months How will we measure completion? endorsement of metrics by RChain president, executive committee, Task Approval Committee, and/or RChain Board of Directors

cc @deannald @patrick727 @kennyrowe @lapin7 @leithaus

Legal

Task Submitter shall not submit Tasks that will involve RHOC being transacted in any manner that (i) jeopardizes RHOC’s status as a software access token or other relevant and applicable description of the RHOC as an “asset”—not a security— or (2) violates, in any manner, applicable U.S. Securities laws.

pmoorman commented 6 years ago

Spoke with @kitblake just now, and he suggested we (him and me) maybe could help with this.

dckc commented 6 years ago

@lapin7 I just re-read your comment; which "we" needs to create metrics? Did Greg take the ball on that part too?

The first I heard of metrics for the bounty system was in the May 9 weekly update after board meeting. I'm still standing by for records of that board meeting too; I don't see them in https://github.com/rchain/board/tree/master/2018 .

lapin7 commented 6 years ago

@dckc Yes getting the minutes correct and approved, is a cumbersome thing.

I guess it's up to the Task Approval Committee (@dckc @PatrickM727 @deannald) now to create metrics, evaluation criteria, objectives, KPI's for the bounty system. I don't know if @leithaus comes up with objectives and criteria.

Until now the objective was:

Do whatever you can to get RChain on the rails, according to your own insights.

Apparently this objective needs to be managed. The good thing is that the trust metric solves some of the problems with concern to:

dckc commented 6 years ago

I guess it's up to the Task Approval Committee ... now to create metrics

No, I don't see anything about metrics in the description of the Task Approval Committee. Perhaps you're asking that we take this on? I'll consider it, but without records of the board meeting, we're missing a lot of essential context. Putting something together for the 201806 pay period seems impractical.

I don't know if @leithaus comes up with objectives and criteria.

What do you mean by that? You just wrote "Greg @leithaus will formulate the objectives and evaluation criteria of the bounty program" two days ago.

deannald commented 6 years ago

I agree with @dckc - I don't see anything about the Task Approval Committee taking on this body of work (creating metrics, etc.) nor direction on this from the Executive Committee minutes from this week.

lapin7 commented 6 years ago

I meant to say: @leithaus said that he would formulate objectives and criteria, but I’m not sure that he will actually do it.

allancto commented 6 years ago

@kennyrowe can we have a rough summary of what the board discussion was on Jun15 (complete minutes would be great, but summary relative to bounty will do)? @lapin7 @deannald @pmoorman @patrick727 @dckc @kitblake are you all available at 9am PDT monday june 25 to discuss objectives and metrics? @leithaus (and others) you are invited certainly as well if it fits your schedule. @lapin7 will you set up the zoom?

Here's the document in progress. Contribute early!- we can follow @dckc 's maxim that if the work is done prior to the meeting we can dispense with the meeting. 👍 https://docs.google.com/document/d/1jXm6Ks9OAe-0mgVOT2QtfaVkHndnOodj3uE5FJJXBNA/edit?usp=sharing

deannald commented 6 years ago

@allancto I would prefer to get some clarity and direction on who will be working on objectives, criteria and metrics before we start working on them with the group you mention above.

dckc commented 6 years ago

I'm not available during business hours Monday. Not until Friday.

Also, as noted above, the board minutes are essential context. I don't see how to proceed effectively without them.

dckc commented 6 years ago

@lapin7 I just re-read your Jun 19 comment:

The 201804 and 201805 Bounty Progress Report was received, but the objectives and evaluation criteria were missing.

In what way were they missing? Why did anyone expect them in that report? That's the first mention of objectives and evaluation criteria in issue #678.

lapin7 commented 6 years ago

@dckc @leithaus expected them in the report.

allancto commented 6 years ago

@deannald based on your comment above I changed the title of this issue to "Interim Report". @dckc , @lapin7 I have also edited the task description to include discussion from the comments above.

dckc commented 6 years ago

@ian-bloom thinks he can ping @leithaus about this in the next day or so

dckc commented 6 years ago

@allancto I ran into this issue in another context and was a little surprised to see the "interim report:" prefix in the summary. I remember now... and that would make sense, if github would let us change the author to you (a feature I'm used to from Trac). But I'm the author, so I tweaked it to something in between. Input to the TAC is not my goal. My goal is to get the whole job done, whatever it is.

deannald commented 6 years ago

@ian-bloom - any luck in following up with @leithaus on this issue on Friday?

kitblake commented 6 years ago

@deannald Ian doesn't read his Github mail. He told me to use normal mail or Discord.

@dckc Particularly for this issue, it would be useful to have the minutes of the Board meeting. "...establish Goals Objectives and Metrics for the bounty system" is a huge scope.

deannald commented 6 years ago

@kitblake - I also emailed him as well.

dckc commented 5 years ago

recall from our July 4 meeting (#823):

@allan - ... Will write objectives and metrics based on the stated purpose, though I don’t think we’re meeting them. @dckc: if you write objectives that we’re clearly not meeting, that will advance the discussion.

What news, @allancto ?

dckc commented 5 years ago

@allancto I see you made substantial progress in objectives and metrics sections in the report as of July 20. IOU a review.

deannald commented 5 years ago

@allancto - Me too (owe you a review). I'm out the next 2 days, so probably some time Thursday...

allancto commented 5 years ago

@dckc @deannald i'm available all day today (Thursday), if you need me let me know what time works best. Thanks! -allancto

allancto commented 5 years ago

I am closing this issue subject to the measure of completion listed in the issue statement (endorsement of metrics by RChain president, executive committee, Task Approval Committee, and/or RChain Board of Directors).

Budget requested: 3600 Valuation model: Value should be assessed by TAC and Directors (reflecting value to our Cooperative). %contribution: 82% @allancto 6% @pmoorman 6% @lapin7 3% @jimscarver 3% @dckc

dckc commented 5 years ago

I don't see the endorsements

allancto commented 5 years ago

@dckc thanks. I can't tell from your response whether the TAC will be endorsing it or not. I'll be happy to work to get it endorsed by our board or president if that's your request, but in that case I'd like to mark the issue as ONGOING and get the part that's already completed voted. I've been working on it for the last 6 weeks.

Thanks! -@allancto

dckc commented 5 years ago

The goal here is to come up with something the TAC endorses. Whether we endorse the current draft isn't clear; I haven't finished my review.

It's reasonable to request a reward for a small part of the budget for getting past a blank page, but what looks like the last 20% of the document is often 80% of the work, so most of the budget should be reserved for when the whole job is done.

dckc commented 5 years ago

I changed the estimated duration from two weeks to six months based on the amount of time it's taking to get feedback from the parties involved.

kitblake commented 5 years ago

Given the current examination of the bounty system, this doc, with its sections on metrics and historical achievements, has turned out to be quite timely. Initially I thought it was an 'extra' but it gathers and provides a lot data to work with.

Note that the title includes "Interim Report". Once things take a definitive form, we might bring out an update based on followup and evolution of the approach. Some of the content can also be used for future communication about our innovative and mesh-based bounties solution.

dckc commented 5 years ago

@allancto what I see in the current draft is a bit different from what I expected, but I suppose it's OK.

In our July 4 discussion, I said "if you write objectives that we’re clearly not meeting, that will advance the discussion". So I expected not only metrics ("number of burgers sold") but also targets ("at least 100 burgers per month") so that we could tell whether we were meeting them or not.

I don't see the targets. But I guess I don't need them.

I'm willing to organize reporting quarterly on the metrics in the current draft: number of participants, utilization of communication channels, etc. (To be clear: I'm also content with the level of reporting we are already doing, so I would be OK if this metrics draft were set aside.)

I reviewed the introductory material only lightly. I was going to suggest removing it, but I see it's valuable to at least one reviewer (@kitblake ) so I'm not going to suggest that after all.

I had one oustanding comment regarding "all other human creations, are inevitably created with “bugs” which @allancto has since addressed.

Also: I don't have access to the version history, so I can't tell when it was last revised.

dckc commented 5 years ago

@deannald , @PatrickM727 , I propose we reboot the list of Bounty Task Guides; that is: revoke all certifications an only reinstate each task area only when we are more clearly positioned to "align contributions with goals of the RChain cooperative."

I hope we can execute the reboot a week from today; I think that gives sufficient time for discussion. :+1: :-1: reactions are most welcome.

For example, for Development, I think #273 demonstrates an effective working relationship with the core development team and a basis for budgets based on story points. A few more things seem worth working out before closing that issue turning this area back on: (a) some redundancy for when I'm not around and (b) the documentation that has been pending for ages, and (c) a budget projection. For the projection, I can base it on our two previous reports:

time period Expenses (USD) issue for details
2018 Q2 $24K #850
2018 Feb-Apr $36K #678
2017 Aug - 2018 Jan $10K #678

Background

Aligning contributions with coop goals has long been a challenge; especially the part about knowing what the coop goals are and whom to coordinate with. To date, we have operated on a best-effort basis and accepted the shortcomings.

Discussion in #bounties since RCon3 has included a considerable amount of dissatisfaction with the return on investment in the bounty system. When considering a suggestion to pause the bounty system until after mainnet, at first I thought that went beyond the remit of the TAC, but it occurs to me that if we revoked all trust certifications, it would have the same effect. I don't think it's essential to wait until mainnet, but I do think it's worth raising the bar for making the relevant sociocratic connections, getting mandate, establishing budget criteria, and making projections.

cc @ddayan @AyAyRon-P @jimscarver @JoshOrndorff @allancto @pmoorman

Anyone new to the discussion is encouraged review

and perhaps some older discussions:

jimscarver commented 5 years ago

I voted on the above proposal and suggest we show our sentiment with 👍 👎 👉 etc.

jimscarver commented 5 years ago

Greg suggests we find sponsors in the coop and the partners for bounty activities and guide members to those activities.

Work on wallets, dApps, training, testing, finding, reporting, creating tests and finding the bugs are parimont activities that should comprise a significant part of our budget.

The social ledger and decision tools to be put on rchain when possible was also a significant ask from many at rcon3 in the realms of organized self organization and decision making.

Marketing is a bit of a pickle, Community building and marketing overlap. We need a clear mandate from marketing on what to guide members to do but not preclude other efforts of value.

Greg's view is that we need to guide the system toward "Sponsored" major want's of the coop core members but not preclude other efforts that have significant member support.

dckc commented 5 years ago

All guides, please note my proposal to the TAC above to reboot all labels. You're more than welcome to show your sentiment with :+1: and :-1: or add comments that go beyond just yes / no.

This list from #925 looks like a handy way to notify all of you:

dckc commented 5 years ago

Some other coop memebers have been discussing related options, and several of them joined a Zoom session that I just hosted: @ddayan , @JoshOrndorff , @luigidemeo and @ysgjay.

After some clarification, it seemed that everybody's goals are largely consistent with the proposal above, with an emphasis on the sponsorship aspect that @jimscarver mentioned. There was interest in overall accounting w.r.t. the 1M RHOC allocated in Aug 2017. I noted the accounting work done in #678, #850.

We did some brainstorming around the challenge of @PatrickM727 's availability to directly participate in the TAC.

I agreed to add Ned from the solutions group to the Development sponsors. And @JoshOrndorff expressed interest in guiding in this area. (Perhaps I / we should split Development into Core bounties and Tools bounties?)

barneycinnamon commented 5 years ago

I support this plan. I would be interested in continuing my work as a trusted guide for Governance issues. I would like to identify a sponsor from among the Coop leadership to partner with on that. I have also submitted myself as a candidate for a board seat so that I could take on the task directly of working to build protocols for members involvement in Coop decision making.

ysgjay commented 5 years ago

@dckc Please add me to "Greeter". Amaka (makys) is no longer a greeter

JoshOrndorff commented 5 years ago

I think everyone on the zoom call agreed that @dckc 's proposal to remove all label guides and add them back only as necessary and fruitful along with @jimscarver suggestion for sponsoring were good and necessary first steps. We also kicked around next steps but everyone was prepared to take this first step.

TrenchFloat commented 5 years ago

Sounds good to me, provided we do more than what has been done to notify everyone being rewarded now. I can hear the "What? My certification is gone?!" already :)

dckc commented 5 years ago

If each of us notifies those we have certified, we should be all set.

AyAyRon-P commented 5 years ago

@dckc is there another meeting slated any time soon? I'd like to continue working in/for/with the same labels - Translation and Marketing to help manage RAM expectations, issues requests and deliverables.

dckc commented 5 years ago

@AyAyRon-P another meeting? Not in particular. I expect this will get air time at the next weekly RAM meeting, and I have asked @TrenchFloat to bring it up on my behalf.

Please don't wait for a meeting to get sponsorship connections and such in order for any label areas you're interested in.

allancto commented 5 years ago

I confirm what @JoshOrndorff said, there is consensus to reboot.

I think everyone on the zoom call agreed that @dckc 's proposal to remove all label guides and add them back only as necessary and fruitful along with @jimscarver suggestion for sponsoring were good and necessary first steps. We also kicked around next steps but everyone was prepared to take this first step.

An important part of the first step needs to be a clearer understanding and possibly of what the role of label guides to actually be. We've gained a lot of experience in our 3 months of trialing. I'm linking a document intended for contributions on how to incorporate and evolve the system, including Purpose, Labels, and Mechanism.

Label guides system review 2018_09. Please take a look and add your viewpoints if you are interested in shaping this reboot (permissions are set to "comment" and I will include all suggestions with authorship attribution, thanks @dckc for emphasizing the importance of attribution).

Thanks! -@allancto

dckc commented 5 years ago

@deannald and I met, along with @JoshOrndorff and @ddayan in behalf of @PatrickM727 to review the plan above. We plan to go ahead, effective Weds Sep 19, by deleting all trust certs and only adding them back as we have sponsorship etc. in place.

We also spent a little time working through overdue task proposals and brainstorming about improved documentation.

JoshOrndorff commented 5 years ago

I confirm that Dan's post is correct and reflects my intentions

-----BEGIN PGP MESSAGE-----

owE1kX1QVFUYxpdlneCGoA0ZgsJBRhg+3FWI74QtlgUVRSVALKKze8/uXrj3nuu9
l2WvSRhj05c2m4mAg1O6GPsHiaOEipOUAypfk2Y1yRhCjV/oaNKYCqx2lvT8dc68
z3ne3/u8zrm+Kj8fY8lY5d4LP2X4fOvTb5pvwtW8rHBIkqAVaWWH/FbLnIlVwIx5
CyNyQLZBGRggHyMBAUsyYCRSEkVklgHkaSAiC0vuEuAUwPAy4mUG8xJFLfMeitLT
CPI8ZOlZ8SrAITkBQBbzVlDDyDagX40lW6HI01i0WGY1epqGCuSJGTAhG2QtAFuA
fj2URcZctTY1MRXImHS1M6iGsCEgsEQMTdiOtKD02ZMorBhAG4J0AkAWCwFk7IiU
aQkUIQGsSE8AJgXQiEUyQ1AgywJZrCbTmZFIZvFyYJ5VAKRpb5304YAJmqsAlEAN
AjZI3CSBDIpFycYIAMlmrZeYdDcjLUUREMhKmGhIIAAClpFlFgGZ4RCowWLV/6Yi
rrbaACEX6WpShFIVEERMUiZ/ZxlMImR4ScYiN0tJNkXy54jGjmhAY3M1R+yhN3Et
9Ym6QKPy8VOlLIjUdOhnNhj7O4tqbRvnP1/2HLV3ryGLIE3M9JUkdgU/y11rxpyK
8p/3XLp9fJ7qa7nvafGnL8X5fx4XviHkyaC9KfxK6SmAnhjbLQuPLYhOyVP3tV3T
TeR11z/afFM/kKbL7GjbSh0a528XGvYcTNjtXP5z113X9NX9O1VNjexl2Ke69+bc
YNf2c3Toke+Vi9H8IcNXybGhsDss8ztw6R3/AyX1j0/t2jXRdf2DtcEdsa8wA/dz
OyfW63wd7XHuHR4Hm14b/1eU8/TLdXsKRr/5Y9G2ygfvOV0t0syUGxSXbixxSjPM
w2hTmidmr6s5Z5PuRPOPK9cNHryJdILreJqmtyHVc6fs9VjfwvMrpsuPClfCR+6+
+pmh+/5Z6jVPxZnA8/1pa5JC8gc2Hzk59MZF/UfH2DT1rdaK/Dq/wICkLMOXF2Ia
Lm+JO8lYs23CpFvK+fhx/tJWYTLwV8e2F82rDbF+yeEOx9SNiNrgO7eibsj/cAsH
xSXHh3VZ61ojl2T8vS8qvvza6BD1hfvP+OKR/dFjV90zQeXLcgMac8evtx8FxrDD
DZ7hnkfG0KDUmea83SszFju3/pBk7vowZTLyYdCmlE5rmRjWqwQo41lRkZqu0SZ3
U73r34zsqNMRdcr7TTvf/n1xfe+J226txpEz5OEKD797tqdx4F52YUSueiR0aqmS
nph/ZnA480DoA/8dyxNxcttY1S/TY63qnhz7uZai9LJL+7YkCpoX1vgXeH6rqHz6
Hw==
=M/XA
-----END PGP MESSAGE-----
TrenchFloat commented 5 years ago

I added to Allan's system review doc a section on the proposal from 9/12's RAM meeting to require guides to sync up with coop "higher ups" for their label and talk about it in issues like #925.

To ensure accountability from the label guides that they are regularly aligning bounty system work with coop goals, it has been proposed (9/12 RAM meeting) that a guide from each label post in each “review of X pay period by guides” issue (like #925 for August) a summary or link of their communications with their relevant connection outside the bounty system. “Relevant connections” for each guide are to be determined, but inferred examples would include Medha P. for the Development label, Patrick M. for Marketing, etc.

(Maybe there's a better word than "higher ups". I just mean people that guides would want to talk to to sync up on goals.)

allancto commented 5 years ago

@TrenchFloat yes, that's the direction we should be going. Furthermore in this issue two fundamentally different things seem to be conflated: is this a reboot of label guides or of trustmetric certification? Probably both require a reboot, but the discussions are (imo) totally different.

What are the qualifications for Trustmetric certification? Trustworthiness, understanding of the voting system, understanding of what makes budgets realistic and SMART. What are the qualifications for Label guiding? Connection within our Cooperative, domain specific knowledge of area or "department". Both are important.

JoshOrndorff commented 5 years ago

@allalcto The proposal is that both trust metric and label guides be reset.

allancto commented 5 years ago

@JoshOrndorff yes, dual reboot. The question is the mechanism for each. Once the trustmetric is reseeded there should be certification voting. Is there guidance as to how that will best proceed? Once label guides are rebooted we'll need to make sure we have a good plan for Labels that represent Cooperative activities and requirements, and find contributors to fill those roles. I've seen @barneycinnamon and @ysgjay (and I'm also hoping for @luigidemeo , @ddayan , @Viraculous, yourself and perhaps others to come forward). What's an appropriate venue for discussing requirements and qualifications and volunteers for guides? What's an appropriate venue for discussing requirements and guidelines for voting Trustmetric ratings?

We'll certainly have some discussion tomorrow at RAM, please everyone interested attend and contribute your thoughts on how these should proceed.

Thanks! -@allancto

dckc commented 5 years ago

@allancto writes:

two fundamentally different things seem to be conflated: is this a reboot of label guides or of trustmetric certification?

I don't see it as two different things.

Once the trustmetric is reseeded

reseeded? The seed remains the same: the TAC (@PatrickM727 , @deannald , and myself).

there should be certification voting. Is there guidance as to how that will best proceed?

Yes: https://github.com/rchain/bounties/issues/783#issuecomment-420802890

The requirements for getting certified to vote / guide an area are

making the relevant sociocratic connections, getting mandate, establishing budget criteria, and making projections.

So write all that up for any area you're interested to see operational and if it looks good, the TAC can certify people who agree to that "charter" for the area label. I gave a nearly complete example for Development above.

David405 commented 5 years ago

@dckc @joshOrndorff @jimscarver I am yet to wrap my head around the proposal.

By sponsorship, do you mean support for a work in the bounty, certification for rating or endorsement to become a label guide?

dckc commented 5 years ago

The most direct form of sponsorship is for bounty rewards to come directly out of the budget of some other part of the coop, such as Marketing or the solutions group.

I can imagine other forms. Perhaps the Executive Committee would agree that it's worth allocating up to $X/quarter for work in some area.

In general, I guess sponsorship is when someone in a position to speak for the coop as a whole agrees how a certain amount of money should be spent through the bounty system.

I expect this question is easier to deal with in the concrete than in the abstract. Is there a specific area that you are asking about?

The new labels don't need to be the same size and shape as the old labels, but just using them as an example... I talked about Development already...

I would expect to see some sponsorship from Marketing for anything like these labels: Branding, Community Building, Events, Marketing (duh), Social media, Translation.

The Executive Committee used to be interested in member-site work, but we haven't talked about it for quite some time.

The RChain Technical Literacy / Education areas seem important, but I don't have a clear sense of what part of the coop deals with them. Perhaps the solutions group is interested to sponsor work in this area, @JoshOrndorff ?

Governance is a puzzler. We say right on the homepage "Cooperative governance and a sustainable economic model make RChain a public utility which can be applied to the greatest problems today." and yet I don't know of anybody leading work on Governance. Voting work has petered out somewhat.

Regarding China, I think I saw @ottermagically say the RChain Asia group is likely to organize something separate from this bounty system. I don't know if there is any staff or the like allocated to Africa.

I asked around about Operations; all I learned is that @allancto is covering some of the duties of @lapin7 .

Greeter work seems to have migrated out of the bounty system to the staff. @ysgjay can you help us understand how Greeter work is compensated, if it is?