GHA: Automation of a Repository Activity Digest

[ExperimentsInHonesty]

Overview

We need an automation that open source repository managers can install, in order to understand the activity over a defined period (e.g., a week, month, etc.).

Action Items

• Tasks for Admin staff
  • [x] draft a proposal
  • [x] ask team members to review
  • [ ] add it to the contribution opportunities for members
• Tasks for Members
  • [ ] Identify what value you would get from features already defined
  • [ ] Identify if there are other things you would like the automation to do
  • [ ] Volunteer to build (self assign the issue to yourself)

Resources/Instructions

• GitHub Actions supports Node.js, Python, JavaScript, Java, Ruby, PHP, Go, Rust, .NET, and more.

History

• @ExperimentsInHonesty
  • sourced a potential candidate: https://github.com/abhijeetps/weekly-digest
  • tested functionality
  • identified that it was not functional
  • determined other people had found it to be non-functional since August 2022
  • opened issue on the repo (received no response)
  • discussed options with team
  • drafted this issue to build our own and make it available via the GitHub Actions Marketplace

[ExperimentsInHonesty]

RAD Repository Activity Digest

Overview

Automations that opens a new issue each week on a repository to keep a specific GitHub team updated on the repository activity.

Problem Being Solved

Members of repository teams have a hard time keeping up with all the activity that happens on the repository over a given period of time.

Value Created from Solution

A replicable GitHub Action and documentation for installation and customization, that can be installed by any repository manager.

Stakeholders Impact - who benefits and how?

• Repository Managers: Ability to see an overview of what is happening on the repo via summary issues
• Contributors: Ability to catch up on what has happened by looking a summary issues

Solution (simplified explanation)

Option 1

• Description: Store the activities as they happen and then generate a report by opening an issue when specified
• Files
  • scripts: capture activities as they happen in the repository
  • JSON object: stores activity records, who generated the activity (GitHub handle) and labels if applicable
  • script: opens issue on the repository at X interval with records from JSON since issue last generated
• Pros and cons
  • Pro: unlikely to hit rate limits no matter how busy the repository
  • Con: a lot of little GHAs to write
  • Pro: each GHA would be easy to customize, in the future, if a repo owner wanted to change information collected

Option 2

• Description: Uses the APIs to poll the repository for activity over a specified date period and and opens an issue
• Files
  • script that uses GitHub APIs to pull all the activity for a defined period
• Pros and cons
  • Pro: might take less time to write
  • Cons: potentially subject to rate limiting

Option 3

Your suggestions wanted

Activities

• Issue
  • New issues created
  • Issue Assigned
  • Issue Unassigned
  • Comment left on issue
  • Issue or comment edited
  • Issue or comment liked
  • Issues closed
• Pull requests
  • PRs opened
  • PRs updated
  • PRs merged
• Stargazers
  • repo stars
  • repo forks
• Releases

Technical Details

Installation

Current State "As-is" most likely something manual but could be partially automated

Future Development The “To-be” state

Anticipated technical outcomes

Resources/Instructions How to run the automation and any packages or dependencies

Language Programming language(s)

Platform Where is it deployed (e.g. GitHub, Linux Command Line, Windows Command Line etc)

Automation triggers Time-based (Specify frequency (e.g. 1x/week) Event-based (e.g. someone just created a new GitHub on a repo)

Input required How much manual or custom input is required?

Output What's the expected result?

API (If applicable)

Versions/Updates

Configuration

Usage

Endpoint documentation

[ExperimentsInHonesty]

Define what the summary looks like

New issues created: 5 issue link by who issue link by who issue link by who issue link by who issue link by who

[ExperimentsInHonesty]

How data should look in a csv or whatever data object you use: Date/Time, What, Who, other (URL, Release name)

[JessicaLucindaCheng]

• Going with Option 2 for the solution because we want to be able to use it retroactively.
• Check what GitHub rate limit is and figure out how to not go over it.
  • Suggested strategy to obtain data from within a given time period
    1. Start asking for data at the start date and keep asking until close to reaching rate limit then record/keep track of the last data item's date
    2. Wait a while for rate limit to reset, then find the last data item based on date/timestamp and get the record after it. Then get data until rate limit is almost reached and record/keep track of last data item's date.
    3. Repeat step 2 until all data is gathered.

[JessicaLucindaCheng]

Progress Update

• Availability: at least 6 hours
• ETA: Wed, Jan 11, 2023 for setting up and at least a plan for how to implement it

Tasks

• [ ] Read Creating actions

Consolidated Notes

• What the summary should look like:
  • New issues created: 5
  • issue link by who
  • issue link by who
  • issue link by who
  • issue link by who
  • issue link by who
• How data should look in a csv or whatever data object you use: Date/Time, What, Who, other (URL, Release name)
• Going with Option 2 for the solution because we want to be able to use it retroactively.
• Check what GitHub rate limit is and figure out how to not go over it.
  • Suggested strategy to obtain data from within a given time period
    1. Start asking for data at the start date and keep asking until close to reaching rate limit then record/keep track of the last data item's date
    2. Wait a while for rate limit to reset, then find the last data item based on date/timestamp and get the record after it. Then get data until rate limit is almost reached and record/keep track of last data item's date.
    3. Repeat step 2 until all data is gathered.

Resources

• https://github.com/octokit
• https://docs.github.com/en/actions/creating-actions
• May possibly want to consider using this for testing: https://docs.google.com/document/d/1A2sj5KROv_AD3qTUrPO-YI730hKj2-o1FnE8GE1PUVY/edit

[JessicaLucindaCheng]

Progress Update

• Availability: at least 6 hours
• ETA: Wed, Jan 11, 2023 for setting up and at least a plan for how to implement it

Tasks

• [ ] Read Creating actions

Consolidated Notes

• What the summary should look like:
  • New issues created: 5
  • issue link by who
  • issue link by who
  • issue link by who
  • issue link by who
  • issue link by who
• How data should look in a csv or whatever data object you use: Date/Time, What, Who, other (URL, Release name)
• Going with Option 2 for the solution because we want to be able to use it retroactively.
• Check what GitHub rate limit is and figure out how to not go over it.
  • Suggested strategy to obtain data from within a given time period
    1. Start asking for data at the start date and keep asking until close to reaching rate limit then record/keep track of the last data item's date
    2. Wait a while for rate limit to reset, then find the last data item based on date/timestamp and get the record after it. Then get data until rate limit is almost reached and record/keep track of last data item's date.
    3. Repeat step 2 until all data is gathered.

Notes from https://docs.github.com/en/actions/creating-actions

• About custom actions
  • JavaScript actions
  • If you're developing a Node.js project, the GitHub Actions Toolkit provides packages that you can use in your project to speed up development. For more information, see the actions/toolkit repository.
• Choosing a location for your action
  • If you're developing an action for other people to use, we recommend keeping the action in its own repository
  • If you plan to combine action, workflow, and application code in a single repository, we recommend storing actions in the .github directory. For example, .github/actions/action-a and .github/actions/action-b.
• Compatibility with GitHub Enterprise Server
  • To ensure that your action is compatible with GitHub Enterprise Server, you should make sure that you do not use any hard-coded references to GitHub API URLs. You should instead use environment variables to refer to the GitHub API:
  • For the REST API, use the GITHUB_API_URL environment variable.
  • For GraphQL, use the GITHUB_GRAPHQL_URL environment variable.
  • For more information, see "Default environment variables."
• Using release management for actions
  • Good practices for release management

To ensure your JavaScript actions are compatible with all GitHub-hosted runners (Ubuntu, Windows, and macOS), the packaged JavaScript code you write should be pure JavaScript and not rely on other binaries.

Resources

• https://github.com/octokit
• https://docs.github.com/en/actions/creating-actions
• May possibly want to consider using this for testing: https://docs.google.com/document/d/1A2sj5KROv_AD3qTUrPO-YI730hKj2-o1FnE8GE1PUVY/edit

[JessicaLucindaCheng]

Progress Update

• Availability: at least 6 hours
• ETA: Get a log of new issues created based on a time period by Feb 5, 2023

Plan

• Issues
  • New issues created
  • Issue Assigned
  • Issue Unassigned
  • Comment left on issue
  • Issue or comment edited
  • Issue or comment liked
  • Issues closed
• Pull requests
  • PRs opened
  • PRs updated
  • PRs merged
• Stargazers
  • repo stars
  • repo forks
• Releases

Tasks

• [ ] Read Creating actions
• [ ] Get a log of new issues created based on a time period

Consolidated Notes

• What the summary should look like:
  • New issues created: 5
  • issue link by who
  • issue link by who
  • issue link by who
  • issue link by who
  • issue link by who
• How data should look in a csv or whatever data object you use: Date/Time, What, Who, other (URL, Release name)
• Going with Option 2 for the solution because we want to be able to use it retroactively.
• Check what GitHub rate limit is and figure out how to not go over it.
  • Suggested strategy to obtain data from within a given time period
    1. Start asking for data at the start date and keep asking until close to reaching rate limit then record/keep track of the last data item's date
    2. Wait a while for rate limit to reset, then find the last data item based on date/timestamp and get the record after it. Then get data until rate limit is almost reached and record/keep track of last data item's date.
    3. Repeat step 2 until all data is gathered.

Notes from https://docs.github.com/en/actions/creating-actions

• About custom actions
  • JavaScript actions
  • If you're developing a Node.js project, the GitHub Actions Toolkit provides packages that you can use in your project to speed up development. For more information, see the actions/toolkit repository.
• Choosing a location for your action
  • If you're developing an action for other people to use, we recommend keeping the action in its own repository
  • If you plan to combine action, workflow, and application code in a single repository, we recommend storing actions in the .github directory. For example, .github/actions/action-a and .github/actions/action-b.
• Compatibility with GitHub Enterprise Server
  • To ensure that your action is compatible with GitHub Enterprise Server, you should make sure that you do not use any hard-coded references to GitHub API URLs. You should instead use environment variables to refer to the GitHub API:
  • For the REST API, use the GITHUB_API_URL environment variable.
  • For GraphQL, use the GITHUB_GRAPHQL_URL environment variable.
  • For more information, see "Default environment variables."
• Using release management for actions
  • Good practices for release management

To ensure your JavaScript actions are compatible with all GitHub-hosted runners (Ubuntu, Windows, and macOS), the packaged JavaScript code you write should be pure JavaScript and not rely on other binaries.

Resources

• https://github.com/octokit
• https://docs.github.com/en/actions/creating-actions
• May possibly want to consider using this for testing: https://docs.google.com/document/d/1A2sj5KROv_AD3qTUrPO-YI730hKj2-o1FnE8GE1PUVY/edit