search

InnerSourceCommons / ispo-working-group

Materials for the ISPO working group
https://innersourcecommons.github.io/ispo-working-group/
Creative Commons Attribution Share Alike 4.0 International
22 stars 4 forks source link

Measure Cross-Org Contributions #58

Closed rrrutledge closed 5 months ago

rrrutledge commented 1 year ago

Have a pattern describing this approach - https://patterns.innersourcecommons.org/

JustinGOSSES commented 1 year ago

I'll write up a quick summary here first.

Our enterprise context

We have a central internal team that maintains and operates a Kusto Cluster that holds several databases of metadata from our version control platforms (GitHub & Azure DevOps). This data comes from the version control platforms' APIs. Putting it in Azure Data Explorer enables it to be queried via kusto queries, which enables certain operations to be done that would take much longer if the person who wanted to query had to do all the API calls to get the data. Kusto query language is a variation on SQL queries. There is also data on reporting relationships in another Kusto cluster that allows it to be determined who reports to who.

What we are measuring with "InnerSource Pull Requests"

We are identifying InnerSource pull requests as pull requests where the most common approvers for pull requests in a given repository have a manager they report to who is not the same person as the manager of the person who submitted the pull request. By measuring how many reporting levels you have to go up before the reporting manager is the same, we can determine approximately how far apart in the company the owning team is from the pull request submitter.

Follow-on measurements

We use this data to produce a few other metrics or tools:

The last metric can be combined with string matching to find InnerSource friendly repositories whose title or README has a given string value.

rrrutledge commented 1 year ago

Thank you for this summary, @JustinGOSSES. This work may be a pattern on its own, or it may be an addition to the Introducing Metrics in InnerSource pattern.

rrrutledge commented 1 year ago

Q: Is it useful to share approaches to this measurement if it depends on a company-specific internal metrics store?

Some thoughts:

A lot of the metrics are specific to individual repositories. This sounds like it could be useful. There is a CHAOSS-inspired approach to gathering metrics. There are program-specific metrics and also project-specific metrics. This is related to a portal for exploring project and metrics. Raw contributions are a subset of the overall area of community engagement (e.g. orbit.love/model).

rrrutledge commented 1 year ago

Q: How can we work on this in a tangible way together?

Some thoughts:

Have a repo where we can have an example portal and iterate on it together. This could go in the innersource commons. The idea is that the portal would have metrics and presentation of projects.

A (Google?) doc where we can each put our point of view.

Q: Will we get lost if we try to work on something as big as a portal (instead of something simpler first)?

Let's split this up:

  1. how it's being measured.
  2. how the information is used.

Here are some elements to think about:

  1. why should we measure cross-org contributions?
    • Legal department may need it for tax (transfer pricing?).
    • Operations may need it for estimating engineering efficiency (and reuse).
    • Book the time of what project people are working on.
    • See where the is a gap between reuse and contributions (a gap between them signifies a contribution barrier).
  2. if so - what should those measurements be?
rrrutledge commented 1 year ago

Q: Why do we need this information?

Some thoughts:

Keep an idea on adoption? Provide a convincing argument of something?

dellagustin-sap commented 1 year ago

Out of the discussion around this topic, the following GitHub discussion was created: https://github.com/InnerSourceCommons/ispo-working-group/discussions/60

rrrutledge commented 1 year ago

We have some PRs in the Managing InnerSource Projects handbook to create a place to document this stuff.

rrrutledge commented 1 year ago

We have a place to document our metrics now! Check out https://github.com/InnerSourceCommons/managing-inner-source-projects/blob/master/CONTRIBUTING.md#metrics

dellagustin-sap commented 11 months ago

This might be solved with #74 , now we have a place to document metrics, the Managing InnerSource Projects book. The original topics was not external yet, thus, moved to challenge.

jeffabailey commented 9 months ago

Duplicate of Microsoft metrics, sort of :)

closing

jeffabailey commented 9 months ago

Duplicate of Microsoft metrics, sort of :)

closing

JustinGOSSES commented 5 months ago

@jeffabailey think this is enough of a duplicate to close? Contribution distance is basically the same concept so if you have measures of it, you're measuring cross org contributions https://innersourcecommons.gitbook.io/managing-innersource-projects/measuring/goals/use_gqm