Valuation of an InnerSource Contribution

[rrrutledge]

@tfroment has some thoughts here as well.

[rrrutledge]

@azinshahidi2019 says: Number of commits doesn't mean anything on its own. How many commits are accepted, etc.? Can we make a formula that can then be reused?

[rrrutledge]

@gruetter: Tried to use cocomo valuation strategy in the past. Spot-checking this with effort vs. commit count gets an amount of time per commit, which can be translated to currency for developer time.

[tfroment]

Yes, I don't have a document ready to be shared, but basically, I had to follow this approach in the past:

1. Categorize the Inner Source Assets into 3 or 4 categories (S, M, L, XL - similar to T-shirt sizes) based on the implementation effort required if someone had to implement it from scratch.
2. Identify, among all the Inner Source Programs, the number of Inner Source Assets reused during a given period of time.
3. Estimate the "knowledge value effort" for each new organization that had the opportunity to reuse an Inner Source Asset (i.e., the additional cost of working on an asset that was not coded by oneself or that depends on another team's expertise).
   • Note 1: This part was challenging to estimate, as we had to interview target teams most of the time, and it was not always possible to estimate.
   • Note 2: If it was not possible to estimate, we had a default (pessimistic) estimation based on asset size.
4. Convert the T-shirt sizes into "number of people year equivalent" (Note: Depending on the asset size, the precision of the evaluation could vary significantly).
5. At the program level, we used a huge Excel sheet to estimate the overall time saved by NOT recoding an asset but rather reusing the one made by another team/entity.

What was interesting is that, in the end, it was very easy to demonstrate that the Inner Source Program was highly profitable for the company, even using pessimistic estimates. This is intuitively what we all think, but it's not so obvious when we're dealing with real-life situations. However, there was a significant dependency on the size of the asset, and the larger the assets were: (A) the estimation was more questionable and (B) the ROI was harder to demonstrate because the knowledge value effort could sometimes skyrocket, which makes sense - we've all struggled with the upfront cost of getting up to speed on a large open-source framework (for example) (even if the potential value is also greater).

[rrrutledge]

@tfroment you say that you don't have a document, but you just wrote a simple document. Let's even get this text that you wrote put out there, somewhere. @spier do you think it could fit in to a pattern somewhere? I took a quick glance, but didn't see anything obvious.

[spier]

@rrrutledge this issue is about measuring the business impact/value of an entire InnerSource program, right?

I think you could use @tfroment's approach as a reference implementation for this pattern. https://github.com/InnerSourceCommons/InnerSourcePatterns/blob/main/patterns/1-initial/introducing-metrics-in-innersource.md

Maybe by extending the text of the existing pattern a bit and then attaching another markdown file with a template for implementation the proposed approach e.g. people-years-saved.md or similar.

Haven't thought through the approach in full but looking at the explanation by @tfroment, a little hypothetical calculation would be useful to understand the approach better. Maybe a spreadsheet with some sample projects, their usage, and the resulting calculated figures. (You mentioned a huge excel sheet on the program level. Maybe a tiny version of this would work)

[azinshahidi2019]

@rrrutledge I think what @tfroment has explained and what @spier have shared are both aligned with the general metrics that can be used to measure InnerSource success or level of ROI. However, if I'm not mistaken, we wanted to use this initiative here to figure out " how to praise people for their contribution?", how do to you measure an individual's contribution to give them points, badges and elevate their profile in general.

[spier]

Meta thought: If we did indeed talk about a related but slightly wrong topic in this thread, then we might add more description to the beginning of the thread. Without sufficient context it otherwise becomes hard to impossible for people that haven't been in the meeting to contribute relevant content to the thread.

However in either case I found the content that was shared here interesting, and capturing it in a more permanent form for others to discover and use would be great!

[rrrutledge]

Makes sense. We are organizing a larger effort around metrics in InnerSource. I will take a look at the content here and loop it in to that other structure that we are creating.

[rrrutledge]

@gruetter and @tfroment what is the goal that you have with measuring the value of a contribution? What is the end thing that you are trying to do after you have this valuation?

[tfroment]

hi @rrrutledge : In the context where I conducted this evaluation exercise on the 'value of reuse' of shared Inner Source Assets within the scope of the ISPO, the specific objective was to justify the financial 'return on investment' of the program to the top management. There were no other 'secondary' intentions, such as recognizing the best projects or carrying out specific actions, even though, as a program leader, having this data for other purposes (such as giving awards to the most reused projects) was interesting.

[rrrutledge]

Thanks, @tfroment. I put your scenarios onto this visualazation. Does it look like I got that right?

@gruetter chime in with your end goal as well?

[spier]

@rrrutledge just following along on the issue itself. Did the content from here get moved anywhere else, or did the ISPO WG decide to not pursue this issue any further?

[rrrutledge]

Put into our general metrics work (linked above)