CommonsBuild / IH-intervention

1 stars 1 forks source link

šŸ”„ PRAISEMAGEDDON šŸ”„ #5

Closed Jeff-Emmett closed 3 years ago

Jeff-Emmett commented 3 years ago

Praise Analysis Dashboard

1. Does this proposal address that some categories may be under rewarded and others over rewarded?

Yes

2. Does this proposal address that paid contributors have had a 50-85% reduction to their total number of impact hours?

Yes

3. Does this proposal address that foundational members of the Token Engineering Community may lack recognition for their less visible work?

Yes

4. Does this proposal address the distribution of impact hours in relation to equality metrics such as the Gini Coefficient?

Yes

What interventions are being proposed?

It has been said in TEC meetings that all work is valued equally, yet the distribution of IH as it stands today does not uphold this community value of equal recognition for work that has been done.

photo_2021-06-29_13-35-40

*This proposal will make the case that the praise system was the wrong tool() to put in charge of the initial token distribution of the TEC builders pool, and that we should throw out the results in favor of something that more reasonably recognizes the efforts put into this beautiful endeavor that is the Token Engineering Commons.**

Welcome to šŸ”„ PRAISEMAGEDDON šŸ”„

photo_2021-06-29_13-35-41 (2)

() Note: We think the praise system works great for a kind of loose & fuzzy social recognition or reputation system like CSTK score as used at the Commons Stack, but that it is not an appropriate tool to be used for precise measurements of contributions as it is being used at the TEC - especially without critical human oversight & adjustment of inadequate outputs. The dangers of using this imprecise reward tool are compounded when these rewards have a financial element, as well as forming a portion of the initial governance in the TEC.*

What is the reasoning?

Through the course of our analysis, we found it very hard to justify the extreme distribution found in IH token rewards, even those occurring among similar contributor groups like ā€˜full time unpaid volunteersā€™, some of whom received multiples more IH than others.

Why would this be the case? The analysis data suggests that certain types of activity are rewarded far more prominently than others. In particular, highly visible & high frequency tasks like attending meetings, posting on social media or ā€˜being a great personā€™ garnered a significant percentage of the rewards, as can be seen in the top categories below. (Contrast these amounts with TE Research or TE Experiments, for an understanding of the scale by which these initiatives are undervalued in the TEC. Work in these areas will pay off in spades for the TEC in coming months and should absolutely be recognized and duly represented.)

Capture4

In comparison, those who work heads down on slower paced research or engineering projects (in particular, token engineers) are less visible, are thus dished less praise, and are as a result underrepresented in the overall distribution of initial governance of the TEC builderā€™s pool. We raised this issue in part because of our continued experience of being unable to adequately praise all of the collaborators in the many TE initiatives we are a part of, because of the heavily manual processes in the praise system. On top of that, the sheer amount of praise dished in TEC calls inflates away most of the IH that those TE collaborators would have been awarded anyway.

The outputs of an algorithm can only be as good as its inputs, and we can say from personal experience that large swaths of the TE landscape is being underrepresented in the IH data, including work done by our own paid contributors in the TEC. Some in this community have said that "these rewards are just for building the TEC". But if the TEC only wants to reward themselves for what the TEC cares about (i.e. no intervention), I believe this community will have a harder time encouraging Token Engineers to collaborate in the future, especially in light of how this topic was handled.

photo_2021-06-29_13-35-41 (3) Unfortunately, this sums up a lot of the praise-gate debate.

The choice for 'no intervention' would be tone deaf for the TEC.

A community who is dedicated to valuing work equitably is not served by inequitable distributions provided by untested tools. The raw outputs of experimental tools & processes need human oversight, and our tools and their outputs should never be taken as "systems we cannot question". Not only is this sentiment rather ironic (considering that questioning broken systems is probably the reason many of us have ended up in this space), it is also not an appropriate sentiment for a community committed to responsible Token Engineering.

photo_2021-06-29_13-35-42

With this proposal, we are trying to speak up on behalf of Token Engineers and other researchers who exist 'between the organizations' (and are thus constantly free ridden from all sides). These people are not recognized by the praise system because they don't show up to TEC calls to do the song & dance for praise of their work. Many of these engineers and researchers are the connective tissue between different organizations, and would be a huge boon to have actively engaged in the TEC ecosystem, which we could encourage with a more equitable IH distribution. This proposal seeks to recognize and value those contributors, to encourage the growth of a positive sum, win-win, "infinite game" ecosystem that would be far more healthy and vibrant than the alternative.

123883577-55175600-d917-11eb-84c8-f2b8c431c7d6

Given that the governance distribution of the builderā€™s pool is a set amount, you can think of the IH distributions as a piece of a pie. Anyone who gets a bigger slice for themselves gives a smaller piece to everyone else. This is an important point to note: whenever someone says they deserve more of the pie, they are simultaneously saying that others deserve less.

photo_2021-06-29_15-03-26

Looking back, what could we have done better?

We propose that the TEC would have been much better off creating a kind of ā€˜contributor cap tableā€™, where each level of involvement was categorized into a certain distribution tier. We propose 4 contributor levels, which can be used to roughly group contributors based on approximated level of engagement & contribution: High activity, mid activity, low activity, and long tail collaborator.

We propose that IH tokens would be more equitably distributed by breaking the distribution of tokens per category of contributor:

Capture

With a presumed distribution of 10,000 Impact Hours, 4000 would be split by the highest activity tier, 3000 for middle activity, 2000 for low activity, and 1000 for the long tail. This would ensure that no single group (e.g. highest contributors) has a majority stake in the builders pool of the TEC. If that seems to give too much of the IH to lower tiers of contributors, keep in mind that each tier grows in size significantly. We estimated that about ~25 people would classify as high activity contributors, ~50 as mid activity, ~100 as low activity, and ~200 in the long tail. So if we were to break it out, it would look something like this:

Capture2 Table 1. Impact Hour distribution for the proposed breakdown of rewards. (Note: This is a rough estimate of an allocation that feels more equitable, of course it is open to tweaks and improvements.) (Note 2: This chart is scaled to 1/5 the total number of members, so I didn't have to draw a table with 400 contributors)

We propose awarding unpaid contributors a 50% bonus on their awarded Impact Hours over those who were compensated $100/day for their work (the typical remuneration rate for a TEC contributor). A sample distribution of Impact Hours allocated per this policy is demonstrated in the chart above, with the notable spikes for the additional 50% allocation for unpaid volunteers.

This distribution would leave a wider distribution of active collaborators as top holders, with the high activity members holding 2-3% of overall governance power in the TEC builderā€™s pool each. This is contrasted with no intervention, where a few members of the high activity group hold 5-7% of the builders pool, and others less than a percent. (Reminder: this is no slight to the members at the top, when we stand behind Rawl's Veil of Ignorance - I would say this even if I were at the top of the list)

Conclusion

The solutions that the world truly needs are often often the least politically feasible.

The same is true of climate change, reparations to the indigenous, and any situation where a minority opinion speaks up against a majority who doesn't want to heed their concerns.

We are aware that this proposal is unlikely to be popular among the TEC community. We decided to carry through with this analysis and proposal anyway, because we believe this would be a far more effective and less inflammatory way of running these kinds of community governance initializations in the future.

photo_2021-06-29_13-35-41

Whether the TEC chooses to adopt this framework or not is completely up to you. Do we equally value contributions in this collective endeavor, or do we cling to inappropriate tools & broken systems that do not serve the mission, vision, & values of this new experimental form of democratic economy?

Will we fall prey to the free rider problem within our own borders, or will we rise to the occasion and make amends? Can we raise the bar on equitable initial governance distribution, or will we relegate ourselves to similar oligarchical distributions as existing protocols, because "that's how things are done"?

We're not sure about you, but we came here for something different.

photo_2021-06-29_14-49-13

Draw 25 cards and go vote TEC!

Then let's get on with our lives šŸ˜‚ photo_2021-06-29_13-35-42 (3)


šŸ¤” Take a look at this forum post if you have any questions.

šŸ—³ When you are finished writing, head over to TokenLog and vote for your favourite proposals!

šŸ“† This voting session ends on Thursday, July 1 at 2pm EST / 8pm CET, the start of the weekly TEC Community Call.

MrsBadgerface commented 3 years ago

Hey @Jeff-Emmett Wow. What a clear and compelling proposal. It seems (to me) that the key to creating just and equitable economies and community spaces is the invitation and space to hold up and examine something that might no longer be working and see how we can adjust our next steps to move closer to that just and equitable ideal. As someone new to this community, I understand there may be context and history that is holding weight here that I don't understand - and I also see how destabilising extensive systems change can be when dramatic (though, again, I don't feel I have enough context to understand the weight of the impact these changes might have to the other aspects of this community) - but this makes sense to me.

I'm curious - near the end you speak of 'we', as in 'We are aware that this proposal is unlikely to be popular among the TEC community. We decided to carry through with this analysis and proposal anyway'. Is this proposal coming from more than just yourself? '

AngelaKTE commented 3 years ago

Jeff - thanks for your proposal! I understand the Tier system, just wonder how we could cover TE contribution and contributors that haven't been covered in praise at all - or way too little. "We propose that IH tokens would be more equitably distributed by breaking the distribution of tokens per category of contributor:" - this seems to redistribute existing IH only.

What I think we should make sure is to include critical contributions we missed to praise. "Many of these engineers and researchers are the connective tissue between different organizations, and would be a huge boon to have actively engaged in the TEC ecosystem, which we could encourage with a more equitable IH distribution. This proposal seeks to recognize and value those contributors, to encourage the growth of a positive sum, win-win, "infinite game" ecosystem that would be far more healthy and vibrant than the alternative." Agree!

So how to

GriffGreen commented 3 years ago

@AngelaKTE what about #2's suggestion?

(Copy paste below)

Determining who in the Token Engineering Field deserves to be rewarded with Impact Hours is a very difficult task, and is unlikely to be agreed upon easily. Who wants to make the list? How do we justify why one person gets more than another? This is not easy! But we are in luck! The praise process as it is already organized is a perfect way to do this, in fact it is intended for this exact purpose!

The Praise with the largest IH reward is this one:

Screen Shot 2021-06-21 at 8 09 50 PM

Here are several others that are hefty TE Praises:

Screen Shot 2021-06-21 at 8 31 13 PM Screen Shot 2021-06-21 at 8 43 38 PM Screen Shot 2021-06-21 at 8 36 47 PM

Screen Shot 2021-06-21 at 8 45 47 PM---

The precedent is set already that we can reward Token Engineers for their past work with Praise, the only requirement is that they join our discord (reasonable if we are giving them governance they need to have access to this key source of information) and that some one explain why they deserve to be rewarded.

All we need to do is just agree that we will give a lot of IH in the last Distro where we collect the praise, we usually do ~1200 (and about 1/2 is taken away from paid contributors) maybe this batch we make it double that assuming we have engagement with listing out all the great TE work that has been done already... And of course it would be great to get a TE aware person to come to a quant to help judge how to distribute.... but also people can just be very explicit in dishing praise, even asking straight up for an IH amount if they want!

I can't think of a better way to reward the TE Community for their work.

Jeff-Emmett commented 3 years ago

Hey @MrsBadgerFace, thanks for your kind comments! The proposal was drafted by myself and @jessicazartler (who graciously provided memetic backup), although these views were formulated through many discussions, realizations and critiques of systems & institutional design spanning many months and many researchers & token engineers. The 'we' used throughout the proposal is intended to refer to anyone who feels this proposal represents their viewpoint. There is no coordinated group driving this proposal outside of the voters who feel this proposal brings something important to the praise debate.


@AngelaKTE, this proposal does not plan to redistribute "existing" IH, it proposes a completely new distribution entirely. The 10,000 IH was used to 'tare' the proposal to roughly the same quantity as the existing system, to keep quantities relatively comparable (it was also an easy number for the math).

I believe it would be fairly easy to lump most contributors into 1 of these 4 groupings based on a few metrics and heuristics. (e.g. High activity contributors show up to calls or working groups consistently, are actively engaged in Token Engineering initiatives that directly benefit the TEC.) There may be a few contributors who are edge cases and up for discussion, but all in all I don't think this would be overly difficult to bucket contributors, considering there are only 4 buckets. The 'width' of these buckets (e.g. 25 ppl for high, 50 ppl for mid, 100 ppl for low) reduces the need for a "perfect ordering" of these contributors - as long as they are in the right bucket, it doesn't matter what order in that bucket they are in. This list will of course include everyone currently on the praise list, and could include others if there was a feeling that some people are not on this list entirely (I have a feeling most of them will be on the list, just with a small IH distribution).


@GriffGreen, I am not opposed to using the praise system to award the correct and representative amounts of IH to everyone who deserves it. However, the heavily manual processes of the praise system make this a non-starter for busy researchers who have other important work to do than spend most of their day quantifying thousands of praise entries. The extent to which IH distributions would need to be altered to provide anything close to a reasonably equitable distribution for the amounts of work put in on behalf of contributors would likely prove to be a lot of work, based on my past experience within the praise system.

If there is a praise improvement proposal to rectify the massive overhead of the praise system, such as an "IH wishlist" that can be easily noted in a spreadsheet by busier members of the community, which can be used by praise quantifiers to ensure adequate IH amounts throughout the quantification process - I am all game. If not, I believe this meme captures the gist of throwing more time into using inadequate tools to fix the problems that they created in the first place:

photo_2021-06-29_13-35-42 (2)

Big love to you all, thanks for commenting! šŸ’š

AngelaKTE commented 3 years ago

Thanks @Jeff-Emmett that's helpful! And thanks @GriffGreen for comparing with #2

kelsien commented 3 years ago

This is a super interesting discussion on algorithmic trust, subjectivity, and accountability. Seems to lend itself to broader questions, assumptions, and values about the role and function of people and machines within CS, as well as the question of rewards at hand. Perhaps this framing is a useful way to surface the communities values on this, move forward, and inform future processes and mechanisms - Kelsie

danelsuga commented 3 years ago

Thank you for this logical, intelligent and also TEC value-aligned proposal. So many great points.

Most of all:

"A community who is dedicated to valuing work equitably is not served by inequitable distributions provided by untested tools. The raw outputs of experimental tools & processes need human oversight, and our tools and their outputs should never be taken as "systems we cannot question". Not only is this sentiment rather ironic (considering that questioning broken systems is probably the reason many of us have ended up in this space), it is also not an appropriate sentiment for a community committed to responsible Token Engineering."

ā¤ļø

sembrestels commented 3 years ago

Doing 4 big categories of contributors instead of value each one for their merits using praise doesn't look good to me, even for future commons. I don't think contributors within the same artificial category have been contributing equally, and awarding all of them with the same amount of IH is not recognizing those differences.

In the case we want to make "more" equal distributions in future commons, i would propose to not use tiers but a continuous transformation formula instead, such as it is done with quadratic voting. This way you assure that any increment in individual contributions is matched with an increment on individual's IH, at the same time you recognize with more IH people who start contributing. We still could argue that 1 IH should always worth 1 IH, independently of who earned it, but that would be another debate.

The other big problem we are facing with the praise system is that we transform qualitative incomplete inputs (praise) into quantitative data (IH), and the process doesn't include all the subjectivities. I think this is difficult to fix, and I'm sure we will find better systems to recognize each other's work for the next Commons. I'd like to praise all the people that have been involved in the praise quantification during all this months, it was a tough work, and totally necessary to be where we are at this moment, and be able to criticize the system to make it better.

Jeff-Emmett commented 3 years ago

Hi @sembrestels, thanks for commenting!

I agree with you that grouping all contributors into buckets risks missing out on "over & above" contributions that are made by key contributors, who should in effect be higher on the reward list.

However, I think the existing distribution does an even WORSE job at recognizing these contributions. Many who have made these superhuman efforts (and deserve to be recognized for them) are actually ranked far below people who had diligent attendance at TEC meetings, as an example.

Durgadas even noted in his forum post that "I often just audit some meetings and end up with praise for something I didnā€™t contribute to". I think Durgadas has contributed greatly to the TEC, but by his own admission he feels this process perhaps recognized him for work he was not actually doing. For context, he almost made it into the top 10 of IH holders, putting him in the top 3% of all TEC contributors.

Let's contrast this with the TEC's own program manager, who works night and day to make sure everything is making progress and hitting deadlines - a never ending and tireless job, that involves active participation in every aspect of the TEC. In recognition for this mighty above & beyond effort, the TEC program manager barely made it into the top 30 of the IH holders list, with less than half of the IH in comparison.

Once again I would like to remind the audience that this is not an attack on anyone's inputs to the TEC - in fact, the proposal being made here is to more equally value all of the above contributions, because the status quo is actively underrepresenting much of the heroic efforts of many who helped build this beautiful community.

While I agree with your underlying sentiment Sem, unfortunately when we look at the quantitative outputs of the praise system, it is very hard to believe that it is recognizing any concrete form of value delivery in the TEC. To be brutally honest, the raw outputs of the praise process looks much closer to a "TEC meeting attendance tracker", which primarily rewards unpaid volunteers. It is for this reason that we propose the tiered system alternative - not as a "perfect" tracking tool (indeed, none exists), but as a necessary improvement over the existing contribution tracking system.

Thanks for commenting! šŸ’š

liviade1 commented 3 years ago

Hey @Jeff-Emmett - my proposal suggests that we write a desired IH amount besides each praise for TE's, plus that we use this opportunity to share and promote some of every person's work to help educate the community and ecosystem. https://github.com/CommonsBuild/IH-intervention/issues/3

If there is a praise improvement proposal to rectify the massive overhead of the praise system, such as an "IH wishlist" that can be easily noted in a spreadsheet by busier members of the community, which can be used by praise quantifiers to ensure adequate IH amounts throughout the quantification process - I am all game. If not, I believe this meme captures the gist of throwing more time into using inadequate tools to fix the problems that they created in the first place:

Big love to you all, thanks for commenting! šŸ’š

JuankBell commented 3 years ago

I really like the idea of better distributions, but I feel that the wording used in this proposal evoques a destructive point of view of the praise system "praisemageddon" rather than a constructive scope incorporating what we have. This, rather than including the other, promotes division and polarity. My suggestion to this proposal is that could be improved by using more comprehensive language.

sembrestels commented 3 years ago

@JuankBell I may not agree on the proposal, but I like its name :smile: Let's call things by what they are, it's essential for being able to discuss them properly.

JuankBell commented 3 years ago

I'm sorry! It was a mistake to close, i wanted to comment, and hit "close with comment" on the minute I saw the mistake i reopened this issue.

GriffGreen commented 3 years ago

it worked out ok

LinuxIsCool commented 3 years ago

This proposal is solid. I appreciate the comments as well. The memes make good points.

I think this proposal gives crystal clear reasoning on why an IH intervention can be beneficial to the long term success of TEC. I think the implementation details can be refined, not sure if the 4 tier system is going in the right direction (maybe it is). I also think that we can have a more precise IH deduction corresponding to wages paid as per #8. I also think that we should implement the infinite vesting tokens solution to give full governance influence proportional to IH per person as mentioned very briefly in #2

MrsBadgerface commented 3 years ago

Reading the comments, it's interesting: I appreciate the praise system but I do wonder sometimes at how much I seem to be gaining when I really don't feel I give much of my time and energy here compared to more involved members, especially members sharing their time in complementary communities; especially when that time in these complementary communities seems to strengthen these members ability to be a part of the TEC. In fact, I feel a bit embarrassed that I'm higher up than someone that is likely giving more than I currently am able to.

Knowing the amount of time and energy I currently give to Enspiral as a volunteer, which is probably between 1 - 2 days a week - on top of everything else - and the immense amount of context and insight that time gives me in understanding Enspiral as a community and system, I would find it psychologically and emotionally tough if my time spent at Enspiral actually reduced how much governance weight I had, compared, say, to someone that didn't have such a large workload or spanned numerous communities. And I would feel depressed to be in a community that didn't seem to value both my input and my insight.

I believe that anyone should be able to arrive in a community and be listened to if they see something not working or could be done better. And... our communities depend on people with context and insight to guide these communities.

I don't really know enough to know if @tamarandom's, or @liviade1's, or @jessicazartler's, or @GriffGreen's proposals, or @Jeff-Emmett's here - all of which recognise the need for making some changes to re-align TEC systems to reflect TEC goals - is the right next move. And I understand @sembrestels' strong sense that changing agreements sets a precedent of 'how can we trust in agreements if they get changed?' and might be destabilising. Perhaps (if TEC doesn't already do this), there's an opportunity here to explore time-bound agreements, so both change and trust is systemised.

Jeff-Emmett commented 3 years ago

Thanks for pointing that out @liviade1, I am happy to see other proposals addressing the weak points in the praise process, and suggesting improvements to address them moving forward!


Hey @JuankBell, I appreciate your feedback. Throughout this process, I have attempted to bring rigorous scientific discussion to the TEC, recommending appropriate algorithmic governance policy to help set precedent for other communities who are watching our progress and learning from what we do. The resulting response from some of the TEC stewards (most prominently "we don't want to share 'our' tokens") has felt like the real slap - not just to myself, but to our community Commons, and to the whole discipline of Token Engineering. No one came here to take away anyone's tokens, the accusation is absurd.

I am not proposing that the entire TEC (or even most of it) agrees with these stewards who resist reasonable alterations to the IH distribution. But part of my frustration with others in the TE community is the overriding desire for placation of those who are "upset by the topic", over a frank discussion around the realities of algorithmic governance and the dangers of putting systems ahead of people. As a collective, we have failed to remain objective by applying Rawl's Veil of Ignorance in this experiment, and the effects are clear.

From my purview, borderline sophistic reasoning was employed to shift the Overton window of this discussion far in one direction, which galvanized opposition to alterations of the praise system among large IH holders. This not only had the effect of sidelining this conversation, making it "weird & awkward" - it almost guaranteed that proposed changes would have to be minimal to be politically feasible.

This šŸ”„ Praisemageddon šŸ”„ proposal was created in response - an attempt to truly remedy the underrepresentation of the people left behind by this skewed distribution. It is a pertinent point that many of those underrepresented are Token Engineers (who should be a key piece of the puzzle for this community by now - at least you would think!)

It does not behoove us as a community to be "so nice" that we aren't capable of expressing how we truly feel about these systems - especially via memes, which allow us to discuss difficult & complex topics with humor and clarity. Given the altar that the praise system has been hoisted upon by those who are opposed to the examination and critical analysis of these tools, at this point I would rather see the praise system be replaced with something altogether less inflammatory and divisive.

"Only fools worship their tools." https://twitter.com/_paulaberman/status/1408595239720341506


@sembrestels, as a thought experiment to test your opposition to this proposal - would you still oppose it if you & I were to switch places on the IH distribution? Let's say I was near the top, and you were feeling that your distribution was underrepresented - would you still vote for no intervention, because you believe these past agreements can't be questioned?

From my side in this scenario where we change places, I would still vote for Praisemageddon, despite my being at the top of the list. I would hear your words, give them the space they deserve, and advocate for change on your behalf - because you are my brother. šŸ™ I am curious if your efforts were underrepresented in the output data - what would you do?


@MrsBadgerface - totally agree with time bound agreements! In fact, all of the agreements that have been made so far were pretty much with the caveat "when we need to change it, we will". This was said when we initially reduced all TE work by -95% to prioritize building the TEC. It was said again when we decided to reduce paid contributors by -85%.

Now there is change being suggested, and a handful of whales in this community seem to be deciding that these agreements can't (or shouldn't) be changed, despite ~97% of IH holders being better off under this more equitable distribution.

I suppose we will see whether this experiment is truly "your economy, your choice", or if the technocracy of a few prominent whales gets to decide the fate of the Commons.