mysociety / alaveteli

Provide a Freedom of Information request system for your jurisdiction
https://alaveteli.org
Other
391 stars 197 forks source link

Better spam handling in (or before) holding pen #1317

Closed crowbot closed 10 years ago

crowbot commented 10 years ago

The volume of spam in the holding pen in WDTK has increased. Come up with some strategies to make this more managable.

hsenag commented 10 years ago

Of 11 spams today so far:

8 were sent "To" the same address, which was a nearly valid old request address - correct hash, but missing the second hyphen.

1 was sent "To" an invalid request address (nearly correct hash) 1 was sent "BCC" a valid request address

1 was sent "BCC" request@whatdotheyknow.com

If a spam was sent "To" an old valid request address then it would be rejected.

It's not entirely safe to just reject mails to old requests with any hash, because sometimes authorities miss out a digit in the request number, though perhaps simply getting a failure bounce would cause them to check.

In any case that wouldn't trivially catch the most frequent case above as it doesn't have an obvious request number.

My initial thought is that if there was an easy way to set an arbitrary "To" address to auto-reject in the same way as for a valid request address, we could cut down the volume quite substantially with minimal effort.

hsenag commented 10 years ago

I've extended this analysis to the last two weeks. Pretty much the same pattern as above, typically with the same addresses, total count over the two weeks is approximately 50.

One new kind I saw (once only) was multiple valid request addresses in the To header.

crowbot commented 10 years ago

Adding a list of "To" addresses to auto-reject seems like it shouldn't be too hard.

garethrees commented 10 years ago

Should we be using greylisting as a more robust/scalable way of filtering obvious spam?

We should also be able to configure the MTA to reject from known spammers.

There's also spamassassin, but that's probably a bit more heavyweight.

Other links