Pull request analytics action

This GitHub Action measures metrics for developers and/or teams. Reports are generated in issues based on user actions such as opening/closing pull requests, requesting/conducting reviews, opening discussions, and more. The action is designed to provide better insights into team strengths and identify bottlenecks.

Table of Contents

• Motivation
• Metrics
• Getting started
• Using GitHub Enterprise Server
• Configuration Parameters Overview
• Outputs
• Recommendations and Tips
• Troubleshooting
• Privacy and Data Handling
• Usage Limitations
• How You Can Help

Motivation

pull-request-analytics-action addresses several key challenges:

1. Identifying Bottlenecks in Code Review Processes: Easily generate summaries showing where delays occur in the review stages.
2. Tracking Trends in Code Review Processes: Analyze how review dynamics change over time to identify positive or negative trends.
3. Detecting Significant Deviations: Identify metrics that vary significantly among teams and developers, revealing potential areas of concern.
4. Simplifying Analysis of Notable PRs: Provides a list of standout pull requests, helping you focus on the most critical cases.

Overall, this action enables faster and more accurate assessments, leading to better decision-making.

Metrics

All metrics are presented in the form of tables, charts, and lists (Report example). Below, you can see an example of such data.

Lead Time

Displays the time from PR creation to each displayed status. Helps identify bottlenecks in the code review process. Use the timeline value in the SHOW_STATS_TYPES parameter.

gantt
title Pull requests timeline(percentile75) 12/2023 / minutes
dateFormat X
axisFormat %s
section dev1
Time in draft(34 minutes) :  0, 34
Time to review request(17 minutes) :  0, 17
Time to review(3 hours 34 minutes) :  0, 214
Time to approve(7 hours 32 minutes) :  0, 452
Time to merge(14 hours 9 minutes) :  0, 849

section dev2
Time in draft(27 minutes) :  0, 21
Time to review request(12 minutes) :  0, 20
Time to review(4 hours) :  0, 240
Time to approve(4 hours) :  0, 240
Time to merge(23 hours 1 minute) :  0, 1381

section dev3
Time in draft(27 minutes) :  0, 15
Time to review request(12 minutes) :  0, 18
Time to review(15 hours 16 minutes) :  0, 916
Time to approve(24 hours 7 minutes) :  0, 1447
Time to merge(53 hours 43 minutes) :  0, 3223

section total
Time in draft(27 minutes) :  0, 27
Time to review request(12 minutes) :  0, 18
Time to review(4 hours 21 minutes) :  0, 261
Time to approve(7 hours 36 minutes) :  0, 456
Time to merge(26 hours 14 minutes) :  0, 1574

pie
title Review time total 12/2023
"0-1 hours(12)":12
"4-6 hours(7)":7
"6-9 hours(4)":4
"12+ hours(2)":2

Contribution

Shows the total volume of code merged, reviews conducted, and comments in PRs. Helps to understand the context in which other metrics apply. Use the workload value in the SHOW_STATS_TYPES parameter.

Discussion Intensity (Author’s Perspective)

Measures how discussion-heavy PRs are from the author's perspective, based on open discussions, review statuses, and the number of comments. Additionally, you can track discussion topics and user agreement by adding discussion topics in [[]] and using thumbs up/down ( :+1: / :-1: ) reactions on the opening comment. Use the pr-quality value in the SHOW_STATS_TYPES parameter.

pie
title Discussions types total 12/2023
"Bug(12)":12
"Performance(8)":8
"Code complexity(3)":3
"Test coverage(2)":2
"Formatting(9)":9

Discussion Intensity (Reviewer’s Perspective)

Measures how discussion-heavy PRs are from the reviewer's perspective, based on discussions, comments, and PR statuses. Helps understand reviewer engagement and decision-making. Use the code-review-engagement value in the SHOW_STATS_TYPES parameter and add thumbs up/down ( :+1: / :-1: ) reactions on opening comments.

Reviewer Response Time

Shows how quickly reviewers respond to review requests. Helps better understand lead time metrics and reviewer engagement. Use the response-time value in the SHOW_STATS_TYPES parameter.

Metric Trends Over Time

This section displays metric changes over time using graphs, helping to understand how metrics have evolved over extended periods. To enable these graphs, ensure that PERIOD_SPLIT_UNIT is set and that the collected data covers at least two time periods (e.g., quarters or months).

$$\color{dimgrey}Time\ In\ Draft\ \color{firebrick}Time\ To\ Review\ Request\ \color{gold}Time\ To\ Review\ \color{chartreuse}Time\ To\ Approve\ \color{blueviolet}Time\ To\ Merge\ \color{orange}Time\ From\ Initial\ Request\ To\ Response\ \color{violet}Time\ From\ Opening\ To\ Response\ \color{mediumblue}Time\ From\ Rerequest\ To\ Response$$

---
config:
    xyChart:
        width: 900
        height: 600
    themeVariables:
        xyChart:
            titleColor: "black"
            plotColorPalette: "dimgrey, firebrick, gold, chartreuse, blueviolet, orange, violet, mediumblue"
---
xychart-beta
    title "Pull request's retrospective timeline(75th percentile) total"
    x-axis ["4/23", "5/23", "6/23", "7/23", "8/23", "9/23", "10/23", "11/23", "12/23", "1/24", "2/24", "3/24", "4/24", "5/24", "6/24", "7/24", "8/24", "9/24", "10/24"]
    y-axis "hours" 0 --> 47
    line [0, 0, 0, 0, 0, 0, 0.13, 0.12, 0.13, 0.42, 0.23, 0.32, 0.15, 0.08, 0.15, 0.18, 0.13, 0.1, 0.17]
line [0, 0, 0, 0, 0, 0.22, 0.13, 0.12, 0.13, 0.32, 0.23, 0.32, 0.18, 0.08, 0.15, 0.2, 0.13, 0.1, 0.17]
line [0.77, 0.65, 1.52, 2.35, 1.42, 2.52, 3.2, 2.13, 4.7, 2.87, 5.95, 4.75, 5.85, 4.98, 3.05, 2.17, 2.5, 3.58, 5.28]
line [2.28, 4.95, 4.1, 4.6, 4.07, 3.3, 6.82, 5.65, 6.72, 4.08, 6.77, 10.43, 7.18, 9.58, 6.9, 4.2, 7.13, 6.35, 8.05]
line [21.52, 28.9, 23.47, 21.2, 23.63, 24.9, 20.72, 29.22, 26.07, 25.52, 22.33, 46.33, 23.43, 26.47, 17.22, 24.28, 21.32, 22.97, 21.95]
line [0, 1.67, 2.62, 3.8, 2.33, 3.15, 4.8, 2.72, 4.9, 2.6, 5.55, 6.12, 5.75, 5.82, 2.98, 1.68, 2.95, 3.92, 5.6]
line [0.5, 0.75, 2.07, 2.28, 1.4, 2.98, 5.17, 2.52, 4.93, 3.57, 6, 7.22, 6.33, 5.77, 3.62, 2.75, 3.28, 3.9, 5.35]
line [0, 2.18, 0.92, 0.77, 5.47, 0.83, 4.85, 2.42, 4.28, 23.18, 0, 1.63, 1.98, 4.13, 1.32, 1.85, 1.63, 2.5, 6.72]

Correlation Between Lead Time and Pull Request Size

This graphs allow you to observe how pull request size impacts lead time. It can be especially useful for assessing the actual influence of PR size on lead time. For more accurate results, it's recommended to analyze a sufficiently large dataset to minimize error margins. To view the graphs, set the SHOW_CORRELATION_GRAPHS parameter to true.

$$\color{gold}Time\ To\ Review\ \color{chartreuse}Time\ To\ Approve\ \color{blueviolet}Time\ To\ Merge$$

---
config:
    xyChart:
        width: 900
        height: 600
    themeVariables:
        xyChart:
            titleColor: "black"
            plotColorPalette: "gold, chartreuse, blueviolet"
---
xychart-beta
    title "Pull request's time/size graph(90th percentile) total"
    x-axis ["xs", "s", "m", "l", "xl"]
    y-axis "hours" 0 --> 95
    line [2.45, 4.27, 6.38, 6.67, 11.53]
line [3.27, 9.05, 14.88, 20.52, 43.67]
line [16.25, 30, 43.25, 54.65, 94.35]

Peak Activity Time

The graph shows when users perform key actions such as opening, closing, and reviewing PRs. To view the graphs, set the SHOW_ACTIVITY_TIME_GRAPHS parameter to true.

$$\color{black}Opened\ \color{purple}Merged\ \color{green}Approved\ \color{red}Changes Requested\ \color{blue}Commented$$

---
config:
    xyChart:
        width: 900
        height: 600
    themeVariables:
        xyChart:
            titleColor: "black"
            plotColorPalette: "black, purple, green, red, blue"
---
xychart-beta
    title "Activity total"
    x-axis ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23"]
    y-axis "Amount" 0 --> 126
    line [2, 4, 0, 0, 1, 0, 0, 4, 7, 22, 50, 61, 66, 91, 82, 85, 93, 75, 58, 35, 13, 2, 3, 2]
line [1, 0, 0, 0, 0, 1, 3, 1, 2, 8, 43, 97, 126, 85, 59, 70, 74, 76, 44, 22, 7, 4, 2, 2]
line [1, 0, 0, 0, 0, 0, 1, 1, 6, 19, 55, 54, 62, 60, 83, 44, 40, 44, 29, 19, 2, 1, 0, 0]
line [0, 0, 0, 0, 0, 0, 1, 1, 2, 7, 17, 20, 14, 12, 9, 9, 7, 12, 9, 2, 0, 0, 1, 0]
line [0, 0, 0, 0, 0, 0, 0, 0, 2, 4, 11, 8, 9, 11, 10, 9, 6, 5, 6, 1, 1, 0, 0, 0]

List of Notable PRs

Identifies standout pull requests, helping quickly locate the most pending PRs at various stages, the largest and the most commented ones. This facilitates analysis by focusing on the most significant cases. Here is an example of the most commented PRs.

1. Feature: PR Title 1(example)(31)
2. Feature: PR Title 2(example)(27)
3. Feature: PR Title 3(example)(25)

Getting started

To integrate pull-request-analytics-action into your GitHub repository, use the following steps. The provided code is a template and can be adjusted to fit your specific requirements:

1. Navigate to the .github/workflows directory in your repository.

2. Create a YAML file, for example, pull-request-analytics.yml.

3. Open your new YAML file and paste the following example workflow. This is a starting template and you can modify it as needed:

   name: "PR Analytics"
   on:
    workflow_dispatch:
      inputs:
        report_date_start:
          description: "Report date start(d/MM/yyyy)"
        report_date_end:
          description: "Report date end(d/MM/yyyy)"
   jobs:
    create-report:
      name: "Create report"
      runs-on: ubuntu-latest
      steps:
        - name: "Run script for analytics"
          uses: AlexSim93/pull-request-analytics-action@v4
          with:
            GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # In the case of a personal access token, it needs to be added to the repository's secrets and used in this field.
            GITHUB_REPO_FOR_ISSUE: # Make sure to specify the name of the repository where the issue will be created
            GITHUB_OWNER_FOR_ISSUE: # Make sure to specify the owner of the repository where the issue will be created
            GITHUB_OWNERS_REPOS: # Be sure to list the owner and repository name in the format owner/repo
            CORE_HOURS_START: "9:00"
            CORE_HOURS_END: "19:00"
            TIMEZONE: "Europe/Berlin"
            REPORT_DATE_START: ${{ inputs.report_date_start }}
            REPORT_DATE_END: ${{ inputs.report_date_end }}

4. Check your repository settings if you want to publish reports in issues. Go to the repository's Settings, and under the Features section, make sure the Issues checkbox is selected. Additionally, if you are collecting statistics for an organization's repository using a personal access token, ensure that the token has the necessary permissions. To do this, go to the organization's Settings and navigate to the Personal access token tab. Verify that the tokens (classic) have permission to access the repository.

5. Decide on which GitHub event you want to trigger the report generation. You can refer to the GitHub Events Documentation for a detailed understanding of different events. In this example, the workflow_dispatch event is selected to allow the action to be manually triggered multiple times with different parameters. report_date_start and report_date_end can be set before running the action without modifying the code.

6. Depending on your needs, you can use either the GITHUB_TOKEN or a generated Personal Access Token (classic). In this example, we are using the GITHUB_TOKEN, but keep in mind that it won't allow you to collect data from multiple repositories or organizations, nor will it provide data segmented by GitHub teams. If these features are critical for you, create a token with the repo and read:org scopes selected on tokens page. You can read more about tokens in the GitHub Documentation.

7. Configure the parameters to suit your needs according to the Parameters Overview section.

8. Merge the code into the main branch of the repository.

9. Open the Actions tab and select the created action from the left sidebar. In our case, it's PR Analytics.

10. In your repository, go to the Actions tab. Select PR analytics and start it via "Run workflow". Fill in any necessary parameters and execute the action. Depending on the number of PRs, it may take from 1 to several minutes to complete.

11. Open the Issues tab, where you'll find the generated report.

Using GitHub Enterprise Server

pull-request-analytics-action supports integration with GitHub Enterprise Server. To use this feature, you need to set the GITHUB_API_URL environment variable:

1. In your workflow file, define the GITHUB_API_URL under the env key.
2. Set the value to your GitHub Enterprise Server API endpoint.

Example:

env:
  GITHUB_API_URL: http(s)://HOSTNAME/api/v3

This configuration allows pull-request-analytics-action to interface with your GitHub Enterprise instance, enabling you to leverage the full capabilities of the action within your enterprise environment.

Configuration Parameters Overview

Below is a table outlining the various configuration parameters available for pull-request-analytics-action. These parameters allow you to customize the behavior of the action to fit your specific needs. Each parameter's name, description, requirement status, and default value (if applicable) are listed for your reference:

Use these parameters to tailor the pull-request-analytics-action to your project's specific requirements.

Outputs

Below is a table describing the possible outputs of pull-request-analytics-action:

Recommendations and Tips

• Use a Personal Access Token (classic) to generate reports for multiple repositories or to support teams.
• Avoid running multiple actions simultaneously that use the same token. This will help prevent hitting secondary rate limits.
• Utilize the schedule event for optimal report updates. You can refresh the report every few hours or days to avoid exceeding rate limits and to keep the report up to date. You can find an example configuration here.
• To hide individual metrics, specify users in the HIDE_USERS parameter or leave total and GitHub team names in the SHOW_USERS parameter.
• To avoid a long list of title changes when updating an existing issue, it is recommended to set the title yourself using the ISSUE_TITLE parameter.
• You can filter pull requests using labels with the EXCLUDE_LABELS and INCLUDE_LABELS parameters.

Troubleshooting

If you encounter a Not Found error:

• Check the scopes of your personal access token if you're using one.
• Verify that you have correctly specified the owner and repository.
• Ensure that you have access to the specified repository.
• If you're using GITHUB_TOKEN, remember that it only provides access to the repository where the action is running.

You can read more about this in the GitHub documentation.

Privacy and Data Handling

pull-request-analytics-action is stateless; it does not send or store any of the collected data. However, to better understand user needs, fix bugs, and efficiently develop the project, some non-sensitive input parameters are sent to Mixpanel. These data are anonymous and do not provide any information that could identify the project or its data. If you wish to disable parameter data transmission, set ALLOW_ANALYTICS to false.

Usage Limitations

pull-request-analytics-action operates within GitHub's API rate limits and message size constraints, which are generally sufficient for detailed, long-term reporting. However, in rare cases of extremely large datasets, some adjustments might be necessary. For more information, refer to GitHub's documentation on rate limiting. The length of the report generated by pull-request-analytics-action is limited to 65,536 characters due to GitHub Issue size constraints.

How You Can Help

Contributions to pull-request-analytics-action are always welcome, no matter how large or small. Here are some ways you can help:

• Star the Project: If you find pull-request-analytics-action useful, consider giving it a star :star: on GitHub. This helps increase its visibility and shows support for our work.
• Spread the Word: Mention pull-request-analytics-action in your articles, blog posts, and social media. The more people know about it, the better it gets.
• Contribute to the Code: Follow our contribution guidelines to make code contributions. Every pull request helps!
• Report Bugs: Encountered an issue? Please let us know by opening an issue on GitHub. This is crucial for continuous improvement.
• Share Ideas: Have ideas on how to improve pull-request-analytics-action? Open an issue and tell us about your suggestions.

I appreciate any contributions to the project. Your help makes this action better!