ayewo
commented
3 years ago I kinda agree with your concerns. In fact, when I read the activities and how they were weighted, it instantly reminded me of organizations that measure developer productivity using a very poor proxy: the number of JIRA tickets closed.
Places that do that will end up elevating developers that appear to be highly productive i.e. they close an average of 15-20 trivial tickets, while they scold developers that appear to suffer from low productivity i.e. they close an average of 1-2 tickets in a week. Only well-run organizations avoid the mistake of blindly comparing JIRA tickets since they understand the fact that JIRA tickets are not equal in terms of effort.
Opportunity Costs
I think a better way to approach this would be to think of contributions in terms of opportunity costs.
Using the example of attending a conference, there are a number of conscious decisions taken by an intending attendee:
- allocate a significant amount of time for booking, travel, conference attendance etc;
- allocate a non-zero amount of money for accommodation, travel, feeding etc (unless the attendee is sponsored by the organizers);
- forgo other activities around the time of the conference or personal purchases that the money could have gone to.
The 3rd decision is the opportunity cost that is often invisible to conference organizers. The attendee is signalling that they would rather be at your conference, even though it is at a significant personal expense to them, than be at somewhere else. This is an enormous sacrifice that deserves more than a weight of 1 or 3.
Proposal
Two suggestions:
- Rename the term "love" to something more neutral like "affinity":
- I think it would be better to split the love metric in four dimensions:
(I know it will make your calculations and the product slightly more complex, but in the long run, it will make your numbers more honest and your suggestions more actionable.)
| Monetary cost | Low | High |
|---|---|---|
| Activity | Answered a forum question | Attended our conference |
| Channel | Online | Offline |
With this split, it is easier to quantify the effort of a member across offline and online channels. For instance, let's say we have three members:
- member A lives in the same city as the conference venue so they don't have to incur the costs of hotel accommodation but they might still have to pay the conference fee.
- member B has to incur hotel, travel and conference fees because they live in a different US state from the conference venue.
- member C is not physically present in the US to be able to attend the conference due to the costs, but they could still opt to attend a live streamāif one is availableāor opt to watch a recording of the conference talks.
Assuming all 3 members go on to share their take-aways to a much wider audience on Twitter, the current weighting would assign them identical scores in terms of impact even though their personal sacrifices are different. (This ignores differences in follower count, tweet impressions and of course how adept each member is at using Twitter for advocacy.)
I'm sure your customers will want to be able to reward their loyal members in a way that is proportionate to the personal sacrifices that they make behind the scenesāsomething that is often invisible to them but which I think you are trying to make more explicit with Orbit.
joshed-io
li-clement
The weights of each activities are always definitely numbers, how do you decide it, engaging with users or decide by community itself ?
I think in some cases, a forum question can be difficult or simple, sometimes you should spend a lot of time to answer it, but sometimes it's a piece of cake, so how do you think about it?