How would you feel about having PR creators in the mix?

johnnyreilly commented 3 years ago

Hey @jeffpriz!

Nice work on this! We've just been taking it for a spin this afternoon and it's really cool!

We were chatting as we were looking at it and wondering about PR creator stats as well. Like, who is contributing most of our PRs that are reviewed? That's super interesting as it lets you know who your big creators are.

How would you feel about having an additional panel in the project for that?

jeffpriz commented 3 years ago

yeah, could be interesting to see who's creating the PRs.. I could see adding some chart/numbers on who's created PRs. Would be an interesting number to also see.

thanks.

jeffpriz commented 3 years ago

I will say, I do have some fear of people mis-using that stat though.. i would hate the idea of a team looking at the list of PR-creators and say "Look Suzy is kicking butt, she added 10 PRs, but Johnny here is slacking at only 3 PRs"...

jeffpriz commented 3 years ago

Sorry for the multiple comments back on this.. but now I'm intrigued.. What if there were a trend-graph of the number of PR creators over time..? So we could get some insight in to the creation activity, without explicitly calling out people PR numbers.. not sure if that's what you're really looking for, but wondering what you think of that idea.

johnnyreilly commented 3 years ago

but Johnny here is slacking at only 3 PRs"...

I find @johnnyreilly is always slacking with his PRs 😅

In all seriousness, this is one of those things that can be used for good or ill. Whenever names and numbers are brought together that's the case; and so that's already the situation with PR approvals and whatnot.

There's potentially intriguing information that can be read out of contributor stats; and it's not just who's slacking and who's not. Consider:

all PRs are not created equal. Seeing that Johnny has done 3 and Suzy has done 10, maybe Johnny is turning out PRs with great code, tests and descriptions, whilst Suzy is doing PRs which contain maybe some code, maybe some tests.... Just more granular.
wait, Johnny and Suzy are crushing it, contributing code and happy. But where's Marvin? Maybe he's struggling to contribute? Maybe we can help him?

Assuming good intent is always a good idea. There are psychopaths out there, but they're the exception and not the rule.

What if there were a trend-graph of the number of PR creators over time..?

This is a different piece of information which is also interesting!

jeffpriz commented 3 years ago

yeah, you're right, I mean there are "good" uses to that data, I'm just always a little sensitive to presenting data that would be looked at "for bad".. (I think it far less likely to have people look "badly" on approval numbers per-person, it just feels far more likely to look at PR creation numbers in a negative light. I would have the same reaction probably if the ask was to show the avg time an approver took to approve a PR). And It feels like if you're just trying to look at who is creating PRs, just looking at the list of PRs in the repository inside ADO can already give you a sense of that.. it's just that assigning of numbers that I get a little leery about I guess...

even the way you phrased the original ask:

Like, who is contributing most of our PRs that are reviewed? That's super interesting as it lets you know who your big creators are.

the idea that you would know who the big contributors are from this is just bad all around right? like you said, PRs are not created equally.. but that wouldn't come across from these numbers.

Assuming good intent is always a good idea. There are psychopaths out there, but they're the exception and not the rule.

yes, I generally like to make the generous assumption... but I've worked with many teams, and tracking numbers like that.. even if there's no bad intent ... just the tracking itself changes peoples behavior. I don't want developers to change their behavior because the team is tracking the number of PRs that they submit

I will think about that one more I think.

johnnyreilly commented 3 years ago

the idea that you would know who the big contributors are from this is just bad all around right?

So context is always helpful. Our organisation has historically used a variety of source control providers. In that mix, Azure DevOps didn't really figure.

We're now tasked with getting the organisation on board, migrating to Azure DevOps and to Azure, and also trying to make sure that good practice is as commonly adopted as possible. The remit of our team is to reach out to teams that might be struggling and help them.

Different teams have different levels of maturity and expertise - and that's fine!

We're not directly interested in this one statistic - we're actually interested in loads! So other things that could be interesting is pipeline success rate. A project that has a high error rate for their pipelines could be a signal that they could do with assistance on building a resilient pipeline.

So we're interested in all kinds of things, with the idea that we might know who we should be trying to help as a consequence.

jeffpriz commented 3 years ago

Thanks for the conversation on this. I hope you understand, I'm not saying it's wrong for you or your team to want to look at this stat, but I'm trying to keep in the context of this is a publicly available extension that at this point 200+ teams have installed. I want this extension to work well in the larger context , so that's why I'm weighing carefully this idea. Here is the blog post I wrote when I published this extension originally: http://blog.oneluckidev.com/post/Pull-request-report I'm just trying to build a case in my head for why we would add PR creators stats for this, and does it fit in how I would want most teams to look at these stats.

thanks again.

johnnyreilly commented 3 years ago

I hope you understand, I'm not saying it's wrong for you or your team to track this, or look at this, but I'm trying to keep in the context of this is a publicly available extension that at this point 200+ teams have installed.

Oh absolutely. That's a tension I'm aware of very well! I work on some open source projects that have a number of users, and knowing what to say "yes" and "no" to when you consider the broader context is very important for ensuring you have a meaningful product with a clear aim in mind. (Whether it's a paid product or not isn't material - the needs are the same.)

Here is the blog post I wrote when I published this extension originally: http://blog.oneluckidev.com/post/Pull-request-report

Yes - I've read it! Great post!

I'm just trying to build a case in my head for why we would add PR creators stats for this, and does it fit in how I would want most teams to look at these stats.

That's fair. I had a question actually: is configuration possible for the project?

Is it possible to say "for repo X, or for project Y I'd like to display these insights"? So say there's a chart that's not interesting to a team, can they opt out of that chart being displayed?

matija32 commented 3 years ago

Hey @jeffpriz , thank you for this plugin, the team got very intrigued. It's a nice dashboard for the team to self-steer their improvements.

We're also for having PR creators as a stat in the dashboard. PR is an atomic change to the codebase and having all information about it helps us have a (more) complete picture of how we are interracting with each other. Right now only one interaction role is shown (reviewer), but other one (creator) not.

Like @johnnyreilly said, it's up to team how to interpret any number. And to be honest, if someone would want to hide some stat in my team, it would be an alarm bell for me that we don't feel comfortable enough to discuss what struggles or distractions we face. That would make me want to have the metric even more :) and that's why we like your dashboard so much!

The power of the plugins such as the one you made is allowing us to have data-centric discussions. It's of course up to you define it's philisophy and what you aim for with it - although making something that fits thousands of team's processes and cultures would just be very hard. So this kind of functionality

"for repo X, or for project Y I'd like to display these insights"?

would allow a wider reach, I guess, but at some points you choose for what kind of team culture you optimize your solution for.

If there's no choice of optionally showing graphs, then I'd always choose for optimizing for a high-performing team. That's the one that seeks out information to validate their impressions on how are they working together, not to look numbers in isolation to make decisions.

matija32 commented 3 years ago

I was thinking even something as simple as checkboxes on top of the page would do the trick.

rosdyana commented 7 months ago

I believe this feature will be great, @jeffpriz do you have plan on this feature? Recently, some devs in my department using PR number as their KPI.

jeffpriz / devops-pr-stats

How would you feel about having PR creators in the mix? #8