There's a huge PR/issue deficit in the Rakudo repo

[JJ]

Only one issue is closed for every 6 open (roughly), and one PR for every two. There are too many PRs, and that will discourage people from sending new PRs. Someone with a stale PR from two years ago will probably never contribute again to the community. So, can you think about some way to deal with this? Clear the backlog of PRs and try to get a bit more up to date with issues?

Branches are also a problem. There are more than two hundred. That's probably better left to another issue here.

So, some possible ideas to solve that:

• Devote part of the time every release to address and clear old PRs. Maybe devote exclusively a release cycle (maybe the one in the summer?) to review and accept/comment/close old PRs.
• Deal with new issues ASAP: at least label them, vet them (for adequacy, relatedness) and close them if they are not going to be addressed
• Devote one squashathon to them?

Any new idea? What would be the best way to deal with this?

[AlexDaniel]

Is the issue solely with old PRs or should we be discussing old tickets here too?

[JJ]

I would say both, but given the size and the optics PRs would be, for me, a priority.

[AlexDaniel]

OK, there are 74 in total, that's not too many. Some of them are probably outdated, and there are many submitted by rakudo devs. So maybe a good start would be to ping some authors who are on IRC, and see what number of PRs we have after that.

That's not a solution, of course, because even if we get that number down to 0, a year or two from now we'll have that issue again.

[JJ]

Well, it's a start.

[AlexDaniel]

Another thing is that from now on language design issues will be opened here, not in the rakudo repo. Hopefully this will make it easier to focus on actual bugs and other compiler fixes.

[patrickbkr]

Same goes for the MoarVM and NQP repos.

I'm not sure a squashathon can be of direct help here as the job of merging PRs is restricted to the core devs. A squashathon could be useful with doing the legwork before an actual merge. (Do they still compile? rebasing)

[patrickbkr]

I do like the idea to somehow incorporate PR work into the release cycle. The most effective way would probably be to treat unprocessed PRs as blockers. That would be a bad idea though as it would cause PRs to be merged before the release, not afterwards.

[JJ]

There would be too many blockers... There are ~ 70 PRs right now. I think it would be better to devote a "slow" cycle to them, like the one in Christmas, or one of the months in summer. Anyway, taking a look at them from time to time would not hurt...

[patrickbkr]

@JJ True. Turning PRs into blockers now definitely won't be much help (and would probably also offend some people). But in the long run (once we have arrived at a sensible level of open PRs), I do like to have a monthly incentive to do the PR work. I'd say without a general change in procedure to provide such an incentive to do the PR stuff, they'll start piling up again.

[vrurg]

I would like to share my latest thoughts on this subject based on own experience.

For a very long time it is assumed that speed is the main problem of rakudo. I've got a feeling that this assumption is erroneous. Over the last year I was trying to implement a couple of projects for own use. None was completed because every time my code bumps into a bug. Each time workarounding the bug would cost me significant rewrites of the code including changes to the interfaces. Eventually, I came to the conclusion that trying to fix these bugs would be of more use to me. But this is me who is a bit of a special case in some respects.

But let's imagine a decent programmer who somehow became interested in particularly Perl 6. He is not bound to any language and may choose of any alternatives on the market equally well. So, the person got attracted by the really amazing features Perl 6 provides. He starts a test project. A 1000 lines of code later – bug. Ok, he gets around it, then proceeds further – and in another 500-1000 lines another bug strikes.

When I imagine myself in a situation like this – my choice is unambiguous: no time wasting, try another lang. Now, if we add what media and other communities think of Perl on top of it then barely 1 out of who knows how many would resist the impulse and stay with us.

Another thing. Lets imagine that our hypothetical apprentice won't just get around the bug but would report an issue. How many of them are never replied? 306 out of 832 opened as of now. Then he goes further and produces a fix and creates a PR. As of now, 20 out of 80 are never commented.

Isn't it really discouraging? Could it be the reason the community doesn't grow as fast as it could? Perhaps time has came to change focus from new features and speed to reliability?

[JJ]

To be sure, not all of the issues might be real bugs, and not all of the PRs real bug fixes. But again, it really needs to be taken care of. Next squashathon will be devoted to rakudo bugs, maybe we could dig into some of these issues... But that will only put a dent on it. We really need a long-term strategy to reduce this amount of issues...

[AlexDaniel]

@vrurg I agree with your thoughts. When I was a release manager, I've put extra effort into making sure that at least we're not introducing new bugs. As to fixing existing ones, I have no idea how to drive that forward. The existence of the problem-solving repo will likely result in less new features being added, which is a good thing in that regard (right?). Squashathons are great but are not very efficient for rakudo-related work, specifically because existing hardcore devs don't feel like participating, or so it seems. Then there's also grant work, and i already expressed in 2016 that I'd prefer to see more work on reliability rather than performance. On a more positive note, things mentioned in my comment were actually fixed. Moreover, nowadays I'm actually using perl 6 in production without a fear of constant headache and crashes.

Isn't it really discouraging?

It's less than awesome for sure, but what is your proposal? What can we change to make things better? I'm looking for something actionable.

[vrurg]

My point is not about wether they're real or not. It is about newbie reaction to all this. So, the strategy must also do something about returning to people, letting them know that their reports are not thrown into void but considered.

Unfortunately, I know how resourceful the task is. And I know how much people do to react and help and not asking for more. Just maybe change of focus.

[vrurg]

@AlexDaniel the final statement of my previous comment is the only proposal I see. Priority should be given to bug fixing. No other solution, considering the lack of resources. I'm a realist and a pragmatist.

This is also the choice I've made for myself: just picking own tickets and trying to develop fixes for them. If at some point I'll have them all closed then would start iterating over other reports.

[AlexDaniel]

I'm stealing this issue for now. Thing is, rakudo/rakudo is not the only repo that suffers from this, and I'm currently looking into other related problems, so this ticket is somewhat aligned with my current efforts.

[JJ]

You need any help, let me know.

[JJ]

Now that we've had a couple of regular, monthly, cycles devoted to new features, and the PR/issue list has kept growing, could we maybe use next cycle exclusively to PRs? Or issues?

[JJ]

Oldest issues are 2.5 years old. Oldest PR is a whopping 6.5 years old. My proposal is that once we're back on track, reducing that load and addressing them, at least the oldest, would help show responsivity towards the community of users.

[AlexDaniel]

Oldest tickets are actually more than 10 years old. It's just that rakudo issue tracker was blocked before 2.5 years ago?

[JJ]

@AlexDaniel We should start with those ones, yet. I can think about several strategies, like for instance every odd month release is devoted exclusively to bugs and old issues (odd =~ old, get it?). Or one developer exclusively devoted to that, out of the group that forms the core. The problem with so many issues is that the origin of some regression issues that arise might be there, but bigger one is that it's impossible to know them all, so many of them might actually be already solved or simply irrelevant.

[JJ]

Ping? About to reach 1000 open issues in Rakudo...

[AlexDaniel]

Nice, let's celebrate. :partying_face:

[JJ]

Does whoever does it get a prize? :thinking:

[AlexDaniel]

Well, I understand that my reply was a bit too festive, but it's literally my birthday today. On a more serious note, PRs are a bit easier to tackle. All you need is some time and courage, just hunt down PR authors and ask them what can be done about their PRs. So you can start with that.

[JJ]

Happy birthday! If you mean "you" as in "me", I don't have the commit bit in Rakudo. I can still ask, but the final word will not be mine...

[AlexDaniel]

I don't have the commit bit in Rakudo

Invite sent. Now you should be able to take care of tickets and PRs.

Later you can also do this:

https://www.perlfoundation.org/contributor-license-agreement print it out, sign it, scan it in (resize your scan to 250KB or less). Email to trademark -at- perlfoundation.org and don't forget to CC will at coleda.com. PDF: https://github.com/rakudo/rakudo/files/3907678/CLA.pdf

[AlexDaniel]

@JJ also, if you can get your ass back to IRC it'll make some things easier.

[JJ]

During confinement, I have too many open messaging applications and I'm starting to be kind of overwhelmed. But I guess that's an order so I'll have to do that too...

[AlexDaniel]

@JJ it's not an order, we're all volunteers here. But people have been sending messages to you so it's good that you finally received them. Also, things just go quicker on IRC (without email notification spam for every single message).

[niner]

On Dienstag, 3. März 2020 07:24:10 CEST Juan Julián Merelo Guervós wrote:

I can think about several strategies, like for instance every odd month release is devoted exclusively to bugs and old issues (odd =~ old, get it?). Or one developer exclusively devoted to that, out of the group that forms the core.

Both strategies would dictate priorities to volunteers. Sorry, but if we have an issue with rakudo at work and I can solve it, I will do that rather than work on some PR or issue that has never been important enough for anyone to actually do something about it in years.

Why is an issue important just because it's old? Many free software projects come to the exact opposite conclusion: if an issue is old and hasn't been touched in a long while, chances are that it's either not an issue anymore or just not important enough for someone to bother fixing it.

Now PRs are a slightly different story. We're lagging there because we don't have enough (wo)man power to review them and guide the developers towards a mergable result. But to get more core developers who can do so, we need to give new contributors a good experience (including timely answers). It's a bit of a chicken/egg situation.

That said, I can't help but notice that for many of these PRs there was a reaction by core developers (surprisingly often even from me) and there were design and/or implementation issues that needed to be resolved, which the developer never did. I don't know what to do about those. Is someone motivated enough to whip up a patch but lacking the perseverance to push it though a real candidate for a core developer?

[JJ]

On Dienstag, 3. März 2020 07:24:10 CEST Juan Julián Merelo Guervós wrote: I can think about several strategies, like for instance every odd month release is devoted exclusively to bugs and old issues (odd =~ old, get it?). Or one developer exclusively devoted to that, out of the group that forms the core. Both strategies would dictate priorities to volunteers. Sorry, but if we have an issue with rakudo at work and I can solve it, I will do that rather than work on some PR or issue that has never been important enough for anyone to actually do something about it in years. Why is an issue important just because it's old? Many free software projects come to the exact opposite conclusion: if an issue

If it's not been addressed somehow it discourages the person that submitted it to keep working. I mean, the important thing is not that it's old, it's that it's better if we don't allow them to get old.

is old and hasn't been touched in a long while, chances are that it's either not an issue anymore or just not important enough for someone to bother fixing it. Now PRs are a slightly different story. We're lagging there because we don't have enough (wo)man power to review them and guide the developers towards a mergable result. But to get more core developers who can do so, we need to give new contributors a good experience (including timely answers). It's a bit of a chicken/egg situation. That said, I can't help but notice that for many of these PRs there was a reaction by core developers (surprisingly often even from me) and there were design and/or implementation issues that needed to be resolved, which the developer never did. I don't know what to do about those. Is someone motivated enough to whip up a patch but lacking the perseverance to push it though a real candidate for a core developer?

In many cases, it needed a consensus, and it was never reached. Fortunately, we have the problem-solving repository for that thing now. Also, if changes were requested to the OP and not done, it's probably safe to issue a warn and close the PR if it's not heeded.

[AlexDaniel]

IMO if you want anything to be done, it's all about getting other people engaged. Walls of text don't really help in these cases, but starting to do some of the work yourself does.

[patrickbkr]

I think it can be a sign of valuing a contribution to solve a merge conflict oneself instead of requesting it to be done by the PR creator. Especially when PRs have been lying around for long.

I might join in on the PR work in the next days.

[nxadm]

IMO if you want anything to be done, it's all about getting other people engaged. Walls of text don't really help in these cases, but starting to do some of the work yourself does.

You're, of course, right. But the discussion is about prioritizing. With the lack of resources that many communities face, this could mean it could be regular cycles of "PR and bug fixes releases", let's day 1 out of 3 releases. Development of new features can continue of course (the itch simile), but their release may have to wait 1 month sometimes. I even think that adding this kind of bug fixing releasing could be a good idea marketing-wise.

[niner]

On Montag, 4. Mai 2020 10:21:10 CEST nxadm wrote:

Development of new features can continue of course (the itch simile), but their release may have to wait 1 month sometimes.

Why would we delay shipping a useful feature that's done? How would this lead to more motivation for developers?

[nxadm]

Why would we delay shipping a useful feature that's done? How would this lead to more motivation for developers?

Do you think it's a big deal to delay a new feature for 1 month every 3 months only if they happen to be ready just before the "bug-fixing release? (And it will be merged the day after the release.)

If you do, it may be indeed a bad idea and we need to look further.

[JJ]

I see it more as a Return-on-investment kind of thing. People invest on a issue, they see a return, they keep coming. From our end, we invest a little time addressing an issue (might result in a change in code or not), we get more engagement from newcomers; also, if they see the issue load is light, they will be more encouraged to submit issues. Also, we might not need to a full release if it includes only merged PRs and old issues addressed, it can simply be a point release.

[niner]

Let me put it this way: getting releases out was important enough for me in the past few months to work on blockers and fix them (even a Windows only issue, despite me having zero interest in that platform), to work on automating release steps and to prepare MoarVM changelogs. That's because those releases contained things that immediately helped us at work.

If I had to wait for an arbitrary period (note that we still don't do too well on making releases actually monthly), I may have put that effort into packaging unreleased versions of rakudo instead, so we could use those.

[JJ]

Let me put it this way: getting releases out was important enough for me in the past few months to work on blockers and fix them (even a Windows only issue, despite me having zero interest in that platform), to work on automating release steps and to prepare MoarVM changelogs. That's because those releases contained things that immediately helped us at work. If I had to wait for an arbitrary period (note that we still don't do too well on making releases actually monthly), I may have put that effort into packaging unreleased versions of rakudo instead, so we could use those.

OK, I see your point. So maybe we should keep this out of the release cycle and just give them some love from time to time? The main point was not to create an exclusive cycle, it was the love thing.

[niner]

I don't want to be too negative here. We clearly have issues with....our issues :D Noone is served by having our lists of issues and open PRs ever growing. I'm just afraid, that there are no simple solutions here. Free software projects have struggled with this problem since time immemorial. There's always too much work and always too few people doing it. Putting more load or some kind of restrictions on those few people has never worked out well.

We need to encourage people who report issues to submit PRs (sadly, "patches welcome" is not enough) and people who submit PRs to follow through and get them merged, but preferably without sacrificing quality on the way or committing us to badly designed APIs. Really, more people need to do reviews, not just a hand full of core developers. Personally, I'm quite encouraged to give a PR a good look if I see approvals from e.g. @AlexDaniel or @lizmat because then I know that the basics are covered for sure and that smart people have had a look already but e.g. just felt not confident enough in the problem space to merge right away.

We used to hand out commit bits freely (that's how I got mine!). Nowadays the stakes are higher, but maybe we can learn from our own past here?

[JJ]

El lun., 4 may. 2020 a las 10:59, niner (notifications@github.com) escribió:

I don't want to be too negative here. We clearly have issues with....our issues :D Noone is served by having our lists of issues and open PRs ever growing. I'm just afraid, that there are no simple solutions here. Free software projects have struggled with this problem since time immemorial. There's always too much work and always too few people doing it. Putting more load or some kind of restrictions on those few people has never worked out well.

We need to encourage people who report issues to submit PRs (sadly, "patches welcome" is not enough) and people who submit PRs to follow through and get them merged, but preferably without sacrificing quality on the way or committing us to badly designed APIs. Really, more people need to do reviews, not just a hand full of core developers. Personally, I'm quite encouraged to give a PR a good look if I see approvals from e.g. @AlexDaniel or @lizmat because then I know that the basics are covered for sure and that smart people have had a look already but e.g. just felt not confident enough in the problem space to merge right away.

Yep, but a bus factor of two is not too good... Also, in some cases I've seen PRs that were approved by one dev grow stale.

We used to hand out commit bits freely (that's how I got mine!). Nowadays the stakes are higher, but maybe we can learn from our own past here?

Well, I just got the commit bit (thanks @alexdaniel) and am doing this for the time being; I'll try to at least address new issues and check out from time to time stale PRs, as well as solve those that are in my hands to solve. Maybe it would be a good idea, as you say, to hand out a few more so that some more people can do that. In order to review PRs you just need the "triage" bit, also. So maybe we could be a bit more liberal with them...

[AlexDaniel]

To clarify, I sometimes give permissions to specific repositories without adding someone to the organization (without them having to sign a CLA). I want to give people enough permissions so that they can do the work they want to do. Not being able to contribute due to permission issues is extremely lame (especially when there's never enough help in open-source projects). This sometimes includes even Admin level permissions so that people can work with webhooks and stuff like that.

I still expect everyone to submit their CLAs though, and to use the given permissions responsibly and with care. I hope that this doesn't lower the standard by any means, and I'd like to sincerely ask people not to disappoint me. :)

[Kaiepi]

I have a bad habit of opening PRs/issues and forgetting about them. I closed some that I could remember off the top of my head that I don't feel should be merged as they stand now. There are probably more, so if there's a PR or issue I've opened that hasn't had any activity for a long time feel free to ping me about it.

Edit: part of the reason behind this is my GitHub notifications and email are very noisy, so if people post reminders as comments on a PR/issue I don't always catch it. You'd have better luck pinging me on IRC.

[patrickbkr]

@Kaiepi You can show a list of all your PRs when you click on "Pull Requests" in the GitHub.com menu bar at the top.

[JJ]

Around 25 PRs merged and/or closed in the last few days, not a bad thing. So maybe we can close this issue, with a compromise of adding more devs with triage privs when it starts to fail? I'll also commit myself to review PRs and solve those that are in my hands.

[AlexDaniel]

Around 25 PRs merged and/or closed in the last few days, not a bad thing

That said, it's scary af to wake up to a few pages of notifications. I'm sorry to all the devs but please bear with us as lots of contributions always result in that and it's actually a good thing that we should all try to appreciate.

There are things that other projects implement to help with this issue. One that comes to mind is stale bot, but I'd personally veto that thing because of the negative experiences I had in the past (big actual issues with productive discussions being on the edge of getting closed just because people are not actively discussing them all the time). I think we should all do our best to help with as many PRs as we can and eventually we'll get the number of open PRs close enough to 0.

So if you're reading this and you have skills and a bit of time to help, please do! Ping me if you need more permissions.

I agree that we can close this for now. I think later we can have a more focused discussion/ticket regarding just PRs of some specific repos.

[JJ]

I'm sorry, I'm reopening this. There are still 1.3k issues in the old issue tracker, and only those that can be closed because they have spontaneously resolved are actually solved. I know it's boring, it's (for the most part) legacy code, it's probably in a part of the code base you're not familiarized with, but they need to be solved, since some of them have been there for more than 10 years (one of them, at least). I know it's easy to be generous with some other people's time, just throw some of them my way and I'll see what I can do. At least triage them and see if they're still current, or they can be thrown into the stale bin.

[MasterDuke17]

At least triage them and see if they're still current, or they can be thrown into the stale bin.

This is where I could see something like @melezhik's work being really useful. New issues aren't supposed to be closed until there are tests in roast for them, but there could be old issues that have been fixed but we don't know it. So something that automatically checks if old issues have been fixed at some interval (every Rakudo commit might be more frequent than necessary, but at least at every new release) would be nice.

[melezhik]

So something that automatically checks if old issues have been fixed at some interval (every Rakudo commit might be more frequent than necessary, but at least at every new release) would be nice.

RICH - http://rakudist.raku.org/play/rakudo-issues is summary of GH issues / RakuPlay tests for some recent GH issues. I've put activity on hold, but if someone thinks it worth I can resume my work.

RICH runs test scenarios defined ether by

• Users as code snippets (RakuPlay) - https://sparrowdo.wordpress.com/2020/12/03/rakudo-issues-player/

or

• Me as Sparrow tasks - https://sparrowdo.wordpress.com/2020/12/18/chasing-rakudo-bugs-with-sparrow/

We can keep both approaches or stick to only one ... I am free for discussion ...

[vrurg]

@melezhik for the old tasks there hardly be any other activity but automated testing if implemented. What would be useful for the core devs is a kind of report listing fixed issues.

BTW, it's a bit of off topic, but I'd be short: I'd be great if you once write a post/article about Sparrow for real beginners. Most of the time you focus on how to do something, but I'm personally missing the overall picture. First of all, I don't really understand what particular problems it solves. Without this I'm afraid to spend time on it only to find out later I don't need it (possibly).