Add manual override for unsent unblock requests

[alexhaydock]

For some sites, valid unblock requests have been submitted but the requests have never been forwarded to ISPs (mostly due to the user never confirming their email address), however the unblock requests are often legitimate and the sites involved are inappropriately blocked.

Perhaps there could be a button for cancelled unblock requests such as this, allowing an admin to "take ownership" of the unblock request and send it using their own email address.

[alexhaydock]

Would like to hear @JimKillock's opinion on this idea. There are many unsent unblock requests which would be useful to send.

[dantheta]

At the moment, the system will only allow a site to be re-reported if it was blocked, reported, unblocked and then blocked again (there is some incompleteness in this logic). We could allow the re-reporting of sites which have had their reports cancelled.

That way any report that belongs to an unvalidated user can be cancelled, allowing it to be resubmitted as you've suggested.

I don't think we should re-use the existing ISP report records, since the creation dates will be quite old and it will skew responsiveness stats. Additionally, the list of blocking ISPs for a site may have changed.

[JimKillock]

That makes sense.

I would suggest the process should be:

(1) The system at the end of 1 month, six weeks or however long should cancel the request;

(2) The admin user should be able to see a list of cancelled reports for review, and from there

(a) Recheck the blocks are in place;

(b) submit a new unblock request, using a random @blocked.org.uk email with a generic name like 'blocked user'; or

(c) remove it from the list of cancelled reports for review; or

(d) add the domain to the domain blacklist (since inevitably a few of these will be pron).

[JimKillock]

We might want to do this soonish, as we have a lot of attention on the old reports queue, and are seeing things that should have been reported etc.

This to my mind also couples with the BBFC requests idea (that idea has been put in multiple tickets, over a few years I will try to close some duplicates later).

[JimKillock]

@edjw flagging this to you as one we could do before the report launch. It would be helpful to have processed the material that's lingering in the queue.

[JimKillock]

It would be helpful to get this done this cycle, I suspect the problem will become obvious or noticed once the report is done, if certain mistakes / reports are noticed by people.

[JimKillock]

How goes with this one?

[dantheta]

A basic version is online now. The option buttons on the ISP reports control panel page have been condensed into an actions menu for each row, since there wasn't quite enough horizontal space.

There are two new options: "Cancel report and resend", and for reports which are already cancelled, simple "Resend". Selecting thee options will clear the previous ISP report and redirect the user to the normal reporting screens with the previous user's comment already loaded in the text box.

[dantheta]

These options currently work for reports in the 'new' or 'pending' state; a little more adaptation is required to allow the cancellation of reports in the "awaiting user verification" state.

[dantheta]

The cancel & resent options are now available for reports that are awaiting user verification.

[edjw]

Sorry catching up on this. I think some of the features here are done and some aren't. I may be wrong though.

I think this is the way @JimKillock suggested it works and that makes sense to me too.

@dantheta – Could you maybe give me the rundown on which of these features are done and which aren't?

1. Every night / every Sunday night (I don't mind), the system checks if there are any 'Unsent' reports that are older than 30 days. 'Unsent' means reports with the status of new, pending or awaiting user verification.

   --

2. Any 'Unsent' reports that are older than 30 days are a) automatically cancelled, and then b) automatically re-checked to make sure the blocks are still active.

   --

3. Any 'Unsent' reports that are older than 30 days and where the blocks are still active are made available to admin users in a single filter in the "ISP Report Admin" page.

   The filter should be called something like "To Re-report".

   --

4. Then an admin user can see all the reports that could be re-reported. Ideally, we would be able to sort the reports by age descending and ascending.

   The admin user could then: a) submit a new unblock request, b) remove it from the list of sites to re-report, or c) flag the report as abuse.

[dantheta]

I've added a wiki page with the ISP report workflow here:

https://github.com/openrightsgroup/Blocking-Middleware/wiki/ISP-Report-submission-process 1 - already covered. At the moment, the only automated cancellation of reports is done by the resend-verification or when a URL has already been unblocked between report submission and retesting.

2 - not required Cancelling a report when it's in the pending state doesn't make as much sense - the test job is queued, and it will get processed when the attached probe is available (even if the probe has been unavailable for a number of days).

3&4 - partly done A list of cancelled reports is available, though the sort options need to be added.

[edjw]

Ok great. Can we add those options to filter by 'Unsent' and sort by age descending and ascending?

[dantheta]

These last two features have been added. The new filter tab is titled "To resubmit". The sort order link is shown at the top of the created column. The ISP reports listing template has been quite extensively refactored too.