Open spier opened 1 year ago
@zkoppert any recommendations about a useful test run that we could do to find out if the Issue Metrics GitHub Action would be helpful for the Patterns WG?
As preparation for a test run, defining the goal seems like a good activity :)
This could include:
Then we could try to generate some of these metrics, using the issue metrics GHA.
Thoughts?
I totally agree that first finding a goal is great! Listing those out as the top priorities and then finding tools to support that is best. For example, "Goals: Be responsive (respond to pr and issues in a few days or less), 3 pattern improvements every quarter, etc.
Happy to help automate the metrics part once there are clear goals!
@zkoppert I might work on this as part of hacktoberfest
. Would you have a bit of time to help me on this?
Yeah! Definitely!
Awesome @zkoppert.
If you could help me do a first iteration of some metrics on this repo itself, then I am sure I can "learn by doing" and figure things out from there.
.md
files created in the 2-structured folder.md
files created in the 1-initial folder If any of these are complicated, let's skip them. Starting with the easy ones is fine.
Optimally we would track this data on a quarterly basis, as we report quarterly from the patterns WG to the board. The quarters we use are non-standard though. We use these months:
Again, if this makes it complicated, let's just track things on a monthly basis.
Would you have time to create a first version of this, that I can build upon? Or would you prefer to do it the other way around? (i.e. I try to do this, and then ask you for PR review and help?)
I can get started on this! I'll let you know which pieces ended up be hard or awkward and thus skipped. :) I should be able to get started on this next week. I've got it marked on my calendar to remind me. 😄
Opened a pull request to get started on automating the metrics listed above. https://github.com/InnerSourceCommons/InnerSourcePatterns/pull/593 It is ready for review. I'll open a new PR later with more metrics as I get time.
@zkoppert I had to do minor fixes but now the GHA runs through: https://github.com/InnerSourceCommons/InnerSourcePatterns/actions/runs/6473958588
The two issues created were #596 and #597.
Looks like it is working! I can start working on the other metrics (number of new Structured patterns, number of new Initial patterns, number of new Known Instances for an existing pattern) later this week. 🚀
Awesome.
I just realized that all metrics have to work of issues/PRs, right? At least in order to use the issue-metrics GHA.
We do have labels that we use for PRs that work on initial or structured patterns. However with those we would not know if a pattern is new in that state or if it would be an improvement to an existing pattern. We could ignore that difference for now though.
Also one general question about the output of the GHA: Would we always create one new issue for each metric that we want to track?
I can write something that doesn't use issue-metrics GHA in order to get the number of patterns by looking at the files in the repo instead of having to rely on pull requests and labels.
We could merge the outputs of the two GitHub Actions into a single report issue if that is preferred. Let me know!
Let's go for the native issue-metrics behavior only. i.e. only search queries (using PRs/labels). We can still get more fancy later :)
Andy for merging: Yes, getting the report for everything in a single issue would be great. (would also make for a nice documentation addition to the issue-metrics README)
Our InnerSource Patterns are maintained by a working group within the InnerSource Commons, often called Patterns WG for short. That Patterns WG is creating a report about anything that has happened related to our patterns once a quarter, to give the Board of the InnerSource Commons visibility into what is going on. These reports have also prove valuable for the Patterns WG itself as it motivates us to pause and look back over the last couple of months every once in a while :)
As an example, see the 2023-05 report.
In these reports we also include some measurable things like
However as some of these measures were hard to collect, we did not do this consistently in our reports.
Now I read about the Issue Metrics GitHub Action, and was wondering if we could use that to generate some of the metrics from our reports automatically? See also this related blog post about data-driven insights by GitHub.