search

hstove / issue_stats

Analyze and compare how long it takes for Github issues to be closed.
issuestats.com
MIT License
184 stars 13 forks source link

[Feature] Time to first answer #10

Open SBoudrias opened 10 years ago

SBoudrias commented 10 years ago

I think a data that'd be cooler to see is how much time pass between issue being posted and the first answer (answer from anybody and answer from one maintainer of a project).

"Time to close" is cool, but not that relevant when speaking about open source as some issues can be open a long time because they're more time consuming or are more generic/future discussion threads.

sindresorhus commented 10 years ago

:+1: I don't really see "time to close" as a useful metric. There are a myriad of reasons why an issue would be open for a long time or closed immediately regardless of the responsiveness of the maintainers.

KangOl commented 10 years ago

It should be "time to first answer from a collaborator".

KangOl commented 10 years ago

Answer or action (rename issue, add label, set milestone, assign it). Simply an ack of the issue.

StephenOTT commented 10 years ago

I played with these types of metrics in https://github.com/StephenOTT/GitHub-Analytics. It was problematic statistics that often created very unreliable results. I found that you would have to pair time to close with other issue attributes such as a breakdown per label or grouping of multiple labels and even some sort of "context" of a close reason to understand at a aggregate level.

On a per issue basis these kinds of stats can be interesting, but when you aggregate, the stats become problematic.

hstove commented 10 years ago

I'd love to report on 'average first response time by collaborator' as well; I've thought about it and would love to have it. Another awesome related feature could be 'top first responders', so for big projects you can see which collaborators respond the quickest.

The issue that blocked development is API quota limits, which issuestats already runs into occasionally. For a project like Rails, we need to paginate through their issues & PRs. They have 15,000 total, and each API call returns 100 results max, so that's 150 calls, which is fine. But to report on 'first responders', we would have to make an API call for every single issue, meaning 15,000 API calls, which is over the quota limit already.

This is all workaroundable, but would require some extra legwork to be smart about dealing with quota limits. Another approach could be to ask Github for a higher quota limit :+1:

The second blocker for me is determining who exactly is a 'contributor' for a project. Unfortunately the repo#collaborators API isn't sufficient for this, as we would need to know which project contributors have push/pull permissions. This is doable through some mashup of the Organization Teams API, but it's a hell of a lot of complexity for a seemingly simple question. I'd love to know if I'm wrong here and that there is a straightforward way.

Thanks for reaching out everyone, and let's keep the conversation going to hopefully get the feature shipped one day!

sindresorhus commented 10 years ago

Another approach could be to ask Github for a higher quota limit

They're usually pretty nice about upping it if you have a good reason for needing it ;)

jspizziri commented 9 years ago

+1 for first response time and top first responders.