Create email PR bot

[Hainish]

Type: feature request

Create a bot that monitors an IMAP endpoint and given new emails of a certain standard format which includes a git patch, opens up PRs on GitHub.

[pipboy96]

@Hainish Care if I will try to implement this?

[pipboy96]

Is it required to specifically use IMAP or can we use Mailgun/Amazon SES?

[pipboy96]

Also, can we allow a web form instead/in addition to email? Sending emails over Tor is difficult.

[pipboy96]

Should only rulesets be allowed for contribution?

[Hainish]

@pipboy96 absolutely, please feel free to implement! I'm thinking IMAP at first, so that someone can email an @eff.org email address (we use IMAP over TLS). A web form would be fine, but let's make sure there's a Tor-friendly captcha system in place as well, since web forms are spam-bait. We should definitely allow any type of PR, as long as it's valid. Perhaps we could require the subject line include something that indicates we should auto-tag a PR with new-ruleset.

[pipboy96]

We also need a CAPTCHA that works without JS.

[pipboy96]

@Hainish Is Golang fine?

[Hainish]

I'd prefer Node so I can review it

[pipboy96]

@Hainish Okay. Thanks for clarifying.

[pipboy96]

@Hainish Any idea for captcha?

[pipboy96]

My guess is that instead of a CAPTCHA we just need to reject anything that isn't a ruleset file or Git .patch. It's highly unlikely that any spambot that is not specifically tailored for this form will be able to generate any of these.

[Hainish]

@pipboy96 I was also thinking that exact thing.

[pipboy96]

@Hainish should we allow sending ruleset file directly if it's new-ruleset (without having the user deal with .patch)? Also, whose email should we set the git commit email to? Some reserved email? The user's email? We should warn that user email is going to be a part of Git history then.

[Hainish]

@pipboy96 git format-patch includes the author ID as specified to git, this should be sufficient (even if it's sent from another e-mail address).

[pipboy96]

I don't quite like an idea of allowing absolutely any email as git commit author. Also what should be done if ruleset is sent without a patch, as a bare .xml file?

[Hainish]

@pipboy96 currently a user can use absolutely any email as the git commit author on GitHub...

I think submitting an unformatted .xml file encourages people to submit-it-and-forget-it. If there is feedback for the author, they should be able to amend the commits with additional commits and re-submit. That's why I think git format-patch should be encouraged.

[pipboy96]

@Hainish Well, some users have never seen command-line Git or desktop clients, but still they write rulesets using GitHub web UI. Some people don't have desktop computers to run git on. There are countless reasons why a user is unable to use git format-patch.

[Hainish]

@pipboy96 in the past, we've had problems with ruleset submission spam. I think some minimal technical overhead is reasonable for an MVP. We can consider if we'd like to open it up more in the future.

[pipboy96]

I think we should reserve the right to modify sent files before merging too (in case a broken file is submitted).

[Hainish]

@pipboy96 Sure, but first things first: let's just focus on a functional MVP using git format-patch. We can always weigh the costs and benefits of further iterations later.

[pipboy96]

@Hainish Can we do away with web form, only leaving a HTTP endpoint usable over curl (by simply sending .patch files with POST)? We can put a script for that in the repo. The most significant advantage of that is lack of spam problems web forms are prone to.

[Hainish]

@pipboy96 sure, works for me!

[pipboy96]

@Hainish Is it fine if I will work on HTTP endpoint first? Also, what's the software that handles emails? Postfix+Dovecot? If this is the case, I think it's not a good idea to poll ICMP, instead a hook must be set up, like this: https://thecodingmachine.io/triggering-a-php-script-when-your-postfix-server-receives-a-mail.

[Hainish]

@pipboy96 HTTP endpoint first sounds good to me!

[zoracon]

If this still is an interest to build I can always reopen. But with the new ruleset context, I imagine this is less urgent.