[OSS] Pull OSS stats on Repo Activity `Due Nov 6`

[jpvajda]

The assumption is: many New Relic OSS projects are abandoned and we want to evaluate which repositories are suffering from this and which one aren't.

objectives

1 show the how much actually goes on in this public github org. 2 identify the projects where we are doing a good job as an open source organization 3 identify the projects where we are not doing a good job as open source organization. \ 4 decide which projects need more open source support and which one's don't.

Data points to query

• Bad contributor experience (avg PR closing time > 1 month)
• Limited to no active community engagement through online forums, events, etc. (No recent comments in the past 2 weeks)
• Issue creation to resolution ratio is poor
• Issue closing rate, no issues have been resolved in 1 month
• contributor abandonment, no recent PRs merged in the past 6 months.
• it's not clear how own's or maintains the repository.

Existing Dashboards

We have a several existing dashboards measuring this data so it's possible we have some of this data already.

• NR1 Developer Ecosystem Metrics
• Programmability dashboard - Tessen

Acceptance

• [x] Create a new Dashboard called New Relic open source ecosystem
• [x] add dashboards that can visualize this data
• [x] if needed, leverage the github API and custom visualizations to show this data.

[polfliet]

Here is a script I used to pull some stats:

#!/usr/bin/env python3

import requests
import os
import statistics
from dateutil.parser import parse
from pprint import pprint

token = os.getenv('GITHUB_TOKEN', '<TOKEN_HERE>')

def calcClosingTime(query_url):
    more = True
    currentPage = 1
    sum = 0
    totalCount = 0
    times = []

    while more:
        params = {
            "state": "closed",
            "since": "2020-10-08T00:00:00Z",
            "page": currentPage,
            "per_page": 100
        }
        headers = {'Authorization': f'token {token}'}
        result = requests.get(query_url, headers=headers, params=params).json()
        count = len(result)    

        if count > 0:
            for r in result:                
                createdDate = parse(r['created_at'])
                if 'pull_request' not in r: # This filters out PRs from issues, because a PR is also an issue for GH
                    if r['closed_at'] is not None:
                        closedDate = parse(r['closed_at'])
                        closingTime = (closedDate - createdDate).total_seconds()
                        times.append(closingTime)
                        sum += closingTime
                        totalCount += 1
            currentPage += 1
        else:
            more = False

    print(query_url)
    print('Analyzed ' + str(totalCount) + ' - Avg closing time (days): ' + str('{:.1f}'.format(sum/totalCount/3600/24)) + ' - Median ' + str('{:.1f}'.format(statistics.median(times)/3600/24)))
    print()

def calcIssueClosingTime(repo):
    calcClosingTime(f"https://api.github.com/repos/{repo}/issues")

def calcPRClosingTime(repo):
    calcClosingTime(f"https://api.github.com/repos/{repo}/pulls")

repo = "newrelic/newrelic-quickstarts"
calcIssueClosingTime(repo)
calcPRClosingTime(repo)

# repo = "newrelic/infrastructure-agent"
# calcIssueClosingTime(repo)
# calcPRClosingTime(repo)

# repo = "DataDog/datadog-agent"
# calcIssueClosingTime(repo)
# calcPRClosingTime(repo)

# repo = "newrelic/infrastructure-agent-ansible"
# calcIssueClosingTime(repo)
# calcPRClosingTime(repo)

# repo = "DataDog/ansible-datadog"
# calcIssueClosingTime(repo)
# calcPRClosingTime(repo)

# repo = "newrelic/terraform-provider-newrelic"
# calcIssueClosingTime(repo)
# calcPRClosingTime(repo)

# repo = "DataDog/terraform-provider-datadog"
# calcIssueClosingTime(repo)
# calcPRClosingTime(repo)

# repo = "hashicorp/hcl"
# calcIssueClosingTime(repo)
# calcPRClosingTime(repo)

[jpvajda]

https://github.com/orgs/newrelic/insights?period=year

[jpvajda]

I've built a dashboard for this data, so closing as this is done for now.