Override probot/stale defaults, if necessary

kytrinyx commented 7 years ago

Per the discussion in https://github.com/exercism/discussions/issues/128 we will be installing the probot/stale integration on the Exercism organization on April 10th, 2017.

By default, probot will comment on issues that are older than 60 days, warning that they are stale. If there is no movement in 7 days, the bot will close the issue. By default, anything with the labels security or pinned will not be closed by probot.

If you wish to override these settings, create a .github/stale.yml file as described in https://github.com/probot/stale#usage, and make sure that it is merged before April 10th.

If the defaults are fine for this repository, then there is nothing further to do. You may close this issue.

kytrinyx commented 7 years ago

I made a mistake.

This was not explicitly documented in the README, but probot/stale requires the presence of a .github/stale.yml file.

If you want the default behavior, you need to add an empty .github/stale.yml file to this repo, otherwise the bot will not be active.

petertseng commented 7 years ago

Hi! I used https://gist.github.com/petertseng/04edd07d4b35cda2cbc1d8142b6fcc13 for this repo as well.

My goal was to find the issues in the history of this repo that would have been closed by the 60-day inactivity period. Then I wished to answer the question: How many of those issues should have been closed?

What does "should" mean, here? It seems to me that there are various kinds of issues:

Issues where we've decided on a clear plan of action
- Good examples: the good first patch issues.
- These issues should probably stay open until the action is taken, no matter how stale they get.
- Only exception probably is if they get so stale that the action they recommend is no longer relevant to the repo, but the bot does not have the ability to decide that, so I say the bot should never close these issues.
- For these issues, the bot forces extra work on us, by forcing us to apply a label to exempt it.
Issues where there isn't a clear plan of action, perhaps discussions or questions.
- For example, #80 went a long time without activity.
- For these it seems being stale implies not enough consensus to change the status quo and/or the querent is no longer interested in the question.
- For these issues, the bot saves us work.

So in this repo's history, 76/231 issues have been open without activity for 60 days at some point in their history (Intentionally without a #. I don't need all those issues to get a link to this issue since they're not actually related to this issue)

539 xjulia builds are running long/hanging (?)
502 say: canonical-data needs improvement.
491 Update tests of Luhn
482 Improve clock description
469 [general] Ambiguity in the x-common README.md
461 all-your-base: split into phases?
455 Suggest specific exercises that deal with multi-language text handling.
454 Rename flatten array to flatten
447 [perfect-numbers] Description is not clear
441 Update or remove non-ASCII test cases.
429 CryptoSquare: add a test for "perfect square" normalized cipher text
428 Discussion: Non-Ascii test cases.
409 Update implementations of Bowling
404 simple-cipher: fix the description
401 Representation of undefined/exceptional values in canonical-data.json
396 Should queen-attack creation tests be improved?
382 Add a philosophy document to exercises directories?
356 crypto-square examples have trailing spaces stripped, not in README 
336 canonical-data.json standardisation discussion (was: Malformed data?)
335 [secret-handshake]: Clarify readme
326 Should the existing text-based exercises be merged?
325 Idea for new exercise (Polar bear)
323 [hamming] Do we need to restrict this exercise to DNA?
321 Is "write a program" accurate?
318 Potential new test in anagram: Same bytes different characters.
313 Saddle Point: Uses y, x coordinate values, rather than x, y
293 Sublist type clarification
292 Exercise Proposal: Parsing Math expressions
290 hello-world: Consistency of the empty string
279 Deprecate binary, trinary, octal, and hexadecimal in favor of new all-your-base exercise
269 Reorganize the x-common files
268 The protein-translation and nucleotide-codons exercises are the same
266 pangram: missing case-sensitivity edge case
248 Redundant data in anagram.json?
244 [binary-search]: Do not implement it for linked lists
238 Incomplete specification for run-length encoding?
234 Binary search should not require a check for sorted input
230 Add instructions on how to extract canonical data to the Contributing Guide
229 Assemble exercise README including the docs/TESTS.md?
225 Exercise Proposal: HTTP Query string builder
224 Allergies READ ME needs clarification 
223 binary and/or trinary: use non-numeric symbols instead of digits?
222 pangram: missing edge case
212 connect: update readme to clarify that not all games are "fair"
208 palindrome-products: README is confusing
202 triangle: clarify confusion and adjust test suites
198 Improve Sum of Multiples exercise
193 linked-list: Suspicious alternative for functional languages
190 crypto-square implementations disagree with README in many tracks
187 Add a CI verification step for metadata filenames
184 Duplicate Hamming distance exercises
171 secret-handshake has ruby code in the readme
168 Gigasecond: remove birthday?
166 Clock exercise improvements
165 Should rikki- submit all the hello worlds?
148 rna-transcription -- weird non-biological thing slipped into this problem
141 Contributing Guide: Anatomy of an Exercise
140 Contributing Guide: How to write a test generator
139 Contributing Guide: Write about how to extract shared inputs/outputs
137 Contributing Guide: Porting an exercise to a new track.
129 accumulate: confusion and unfortunate naming
110 New About Language Section
107 Sources of ideas for new problems
102 Insights - what data would be helpful in improving the tracks?
 99 Boundary conditions for exercises
 95 Improve test inputs for 'binary'
 94 Exercises about refactoring
 89 queen-attack might be missing test cases
 86 Scrabble-score: 'multibillionaire' is too long for an actual scrabble board
 80 Exercises without a test suite?
 74 wordy: clarify README
 72 A place for track specific metadata in the .yml files
 71 Subtracks, 2.0 exercises, tags
 69 Move language-specific documentation into language-specific repositories
 63 Exercise classification
  1 Remove spurious reference to other exercise in linked-list metadata

So here's where my work is incomplete. In other tracks, I was able to decide: Are the majority of issues those that should be closed or not? I haven't yet done that work for this repo. So you can consider me to have no vote yet.

If it turns out that the majority of these issues were of the type that should be closed, then I would support activating the bot in the repo. If the other way around, I would oppose.

petertseng commented 7 years ago

Hi. The number of issues without activity in 60 days has increased to 107/231. Here is the list of the 31 issues that are now stale.

523 luhn: It's about checksums.
554 bank-account: Implement canonical-data.json
555 binary-search-tree: Implement canonical-data.json
557 diffie-hellman: Implement canonical-data.json
558 dot-dsl: Implement canonical-data.json
559 error-handling: Implement canonical-data.json
560 go-counting: Implement canonical-data.json
561 grade-school: Implement canonical-data.json
562 hangman: Implement canonical-data.json
564 house: Implement canonical-data.json
565 kindergarten-garden: Implement canonical-data.json
566 ledger: Implement canonical-data.json
567 lens-person: Implement canonical-data.json
568 linked-list: Implement canonical-data.json
569 matrix: Implement canonical-data.json
572 paasio: Implement canonical-data.json
573 palindrome-products: Implement canonical-data.json
574 parallel-letter-frequency: Implement canonical-data.json
576 poker: Implement canonical-data.json
578 proverb: Implement canonical-data.json
579 pythagorean-triplet: Implement canonical-data.json
580 robot-name: Implement canonical-data.json
583 scale-generator: Implement canonical-data.json
584 series: Implement canonical-data.json
585 sgf-parsing: Implement canonical-data.json
586 simple-cipher: Implement canonical-data.json
587 simple-linked-list: Implement canonical-data.json
589 tree-building: Implement canonical-data.json
590 twelve-days: Implement canonical-data.json
592 zebra-puzzle: Implement canonical-data.json
593 zipper: Implement canonical-data.json

Now, of these 107 issues, if I can find at least 54 issues that should not be closed, then I will oppose using probot/stale in this repo.

All 30 "Implement canonical-data.json" issues qualify, in my view.

Then, I claim that the following 31 issues qualify:

502 say: canonical-data needs improvement.
491 Update tests of Luhn
482 Improve clock description
441 Update or remove non-ASCII test cases.
429 CryptoSquare: add a test for "perfect square" normalized cipher text
409 Update implementations of Bowling
404 simple-cipher: fix the description
335 [secret-handshake]: Clarify readme
279 Deprecate binary, trinary, octal, and hexadecimal in favor of new all-your-base exercise
269 Reorganize the x-common files
268 The protein-translation and nucleotide-codons exercises are the same
266 pangram: missing case-sensitivity edge case
234 Binary search should not require a check for sorted input
230 Add instructions on how to extract canonical data to the Contributing Guide
224 Allergies READ ME needs clarification 
222 pangram: missing edge case
212 connect: update readme to clarify that not all games are "fair"
208 palindrome-products: README is confusing
202 triangle: clarify confusion and adjust test suites
198 Improve Sum of Multiples exercise
187 Add a CI verification step for metadata filenames
171 secret-handshake has ruby code in the readme
148 rna-transcription -- weird non-biological thing slipped into this problem
141 Contributing Guide: Anatomy of an Exercise
140 Contributing Guide: How to write a test generator
139 Contributing Guide: Write about how to extract shared inputs/outputs
137 Contributing Guide: Porting an exercise to a new track.
 99 Boundary conditions for exercises
 86 Scrabble-score: 'multibillionaire' is too long for an actual scrabble board
 69 Move language-specific documentation into language-specific repositories
  1 Remove spurious reference to other exercise in linked-list metadata

So this tells me that in 61/107 cases the bot forces us to take on extra work and in 46/107 cases it saves us work.

As promised, since the majority of the time the bot creates more work than it saves, I oppose having the bot for this repo.

kytrinyx commented 7 years ago

@petertseng I agree. We should not have probot/stale for this repo.

After having seen the analysis on a number of repos, I think that the only repo we should absolutely have probot on is exercism/exercism.io. In other repos it might be handy, but not vital.

petertseng commented 7 years ago

I wonder if we will wish things were better in those 46/107 cases. After all, 67/107 is only about 5/8 issues. In such a case we would want to think of ways to reduce the amount of work of the 67/107. For example if adding the bot adds 1 unit of work for 67 issues that shouldn't be closed but saves 2 units of work for 46 issues that should be closed, that would be a savings.

I welcome more nuanced analyses than mine at any time. After all, the data showed us there are stale issues, and there is still the question of how to deal with them.

If the answer is "a dedicated and motivated person looks through issues from time to time" then perhaps that person will have more data on how much time the bot would have saved.

This repo, being one shared by all tracks, benefits greatly from things that mitigate the bystander effect.

exercism / problem-specifications

Override probot/stale defaults, if necessary #746