We get a lot of requests for, or containing, personal information

[henare]

This is a problem because it exposes personal information and administrators have to go in and manually hide requests.

I'd like to quantify the problem and then work out what we should do about it. Starting this week I'm going to note down actions we take.

• Week beginning 2016-08-01: 2 requests hidden
• Week beginning 2016-08-08: 2 requests hidden
• Week beginning 2016-08-15: 2 requests hidden
• Week beginning 2016-08-22: 2 requests hidden
• Week beginning 2016-08-29: 3 requests hidden
• Week beginning 2016-09-05: 8 requests hidden
• Week beginning 2016-09-19: 35 requests hidden (research week)
• Week beginning 2016-09-26: 1 requests hidden
• Week beginning 2016-10-03: 11 requests hidden

[henare]

I've opened this upstream issue with an idea of how to fix this https://github.com/mysociety/alaveteli/issues/3515

[henare]

At our performance/feedback hackday we realised that we should create a chart of hidden requests over time, i.e. show this data automatically rather than collecting it manually like this. By doing this we'll be able to see if any solutions we implement are having the desired effect.

[henare]

The first step is to email the list about https://github.com/mysociety/alaveteli/issues/3515. We want peoples' thoughts on that approach and we also want to ask other installs how big of a problem it is for them. We'd like to do this soon as it has lead-time.

[benrfairless]

I know we all hate footers, but what about changing the footer at lib/views/outgoing_mailer/_followup_footer.text.erb to specifically remind the agency not to respond to requests for personal information?

[henare]

@benrfairless that doesn't help the problem of people putting in requests for personal information.

[equivalentideas]

We've just sent that message to the users and dev Google Groups

[equivalentideas]

While we're waiting for peoples’ input, we'll start on the chart:

At our performance/feedback hackday we realised that we should create a chart of hidden requests over time, i.e. show this data automatically rather than collecting it manually like this. By doing this we'll be able to see if any solutions we implement are having the desired effect.

[equivalentideas]

While we're waiting for peoples’ input, we'll start on the chart

We've now done this over here https://github.com/mysociety/alaveteli/pull/3544

Moving on with producing a solution to our problem. The first step is to look at all the hidden requests on our site and see if they are in fact all to a small group of authorities—in which case our solution idea here could be good https://github.com/mysociety/alaveteli/issues/3515

But if the requests are too lots of different authorities, there might be some common text we can target for a solution like Francis suggested: https://github.com/mysociety/alaveteli/issues/3515#issuecomment-253090232

[equivalentideas]

The first step is to look at all the hidden requests on our site

Here's a script to run with rails runner to do just that: https://gist.github.com/equivalentideas/23335c209c3857f2e16049ea78289822

[equivalentideas]

Ran this on the server with:

sudo su - alaveteli
RAILS_ENV=production /home/deploy/.rvm/wrappers/ruby-2.0.0-p353/bundle exec rails runner extract_hidden_requests.rb

And then copied to my machine with scp. I then removed the .csv data and extract_hidden_requests.rb script from the server.

It's on Google Drive for analysis here https://docs.google.com/spreadsheets/d/1M2jUSTDzO4fpKrUzIblAWocjbWcMzXJfFQ6IvY3NoyA/edit#gid=1299707278

[equivalentideas]

Hidden requests by Jurisdiction

Jurisdiction	Number of hidden requests
federal	132
nsw	83
vic	38
nt	5
act	4
qld	3
wa	3
sa	2
Grand Total	270

Over time

[equivalentideas]

Top 20 authorities by number of hidden requests

Authority	Number of hidden requests
NSW Police Force	58
Australian Federal Police	18
Department of Human Services	17
Department of Health	16
Victoria Police	13
Australian Taxation Office	10
Department of Immigration and Border Protection	9
Roads and Maritime Services	8
VicRoads	6
Department of Defence	5
NT Police, Fire and Emergency Services	4
Federal Circuit Court of Australia	4
Australian Postal Corporation	4
Office of the Commonwealth Director of Public Prosecutions	3
NSW Department of Family and Community Services	3
CrimTrac	3
Victorian WorkCover Authority	2
Victorian Road Safety Camera Commissioner	2
SA Health	2
RSPCA Victoria	2

[equivalentideas]

Moving on with producing a solution to our problem. The first step is to look at all the hidden requests on our site and see if they are in fact all to a small group of authorities—in which case our solution idea here could be good mysociety/alaveteli#3515

But if the requests are too lots of different authorities, there might be some common text we can target for a solution like Francis suggested: mysociety/alaveteli#3515 (comment)

From the tables/chart above and from looking at the data we can see that:

1. The frequency of request hides is increasing
2. Requests to state authorities make up a vast, increasing majority of those hidden
3. The top ten authorities for request hides make up 60% of all hides.
4. NSW Police Force, Australian Federal Police, Victoria Police, NT Police, Fire and Emergency Services, Western Australia Police, and Queensland Police Service have been the authority for 95 hides between them, 35% of total hides.
5. 77% of hidden requests were to authorities that have had at least one other request hidden.

So I think our hypothesis that most hidden requests are to a relatively small group of authorities is validated.

[equivalentideas]

A quick filter of the initial messages in these requests shows that 55% contain the string " my ". I just did a quick page search through a few dozen of public requests on the site, and when you filter out the " it is my will" and " my request ", there aren't many with "my" in the initial message at all.

This could be another way to pick up that the person might be making a personal request and help them not make a public request.

[equivalentideas]

After looking through our observations, @Henare and I decided we’ll smash together an experimental implementation of this authority based solution https://github.com/mysociety/alaveteli/issues/3515#issue-180561855 in our theme.

[equivalentideas]

Our idea it to insert something like:

At the top of the form, before the subject:

If no is selected, the person proceeds through the form as normal.

If they select yes, then the rest of the form is replaced with a message that provides some way for people to make their personal request outside of the site.

[henare]

Here's some text I've been drafting:

Are you asking for personal information about yourself?

(*) Yes ( ) No

Please email your request directly to NSW Police Force using:

gipaapp@police.nsw.gov.au

You cannot make requests for personal information using this site. If you do you will be posting your information publicly on the internet for anyone to see.

[equivalentideas]

Here's how we've implemented this:

After this I made the buttons a little smaller, I thought they were dominating the form too much after you've chosen.

[benrfairless]

Does it apply to all requests or just some?

Ben Fairless Volunteer - Right to Know - @RightToKnowAu

A project of the OpenAustralia Foundation

On 27 Oct. 2016, at 12:16 pm, Luke Bacon notifications@github.com wrote:

Here's how we've implemented this:

After this I made the buttons a little smaller, I thought they were dominating the form too much after you've chosen.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[equivalentideas]

Does it apply to all requests or just some?

@benrfairless Only requests to authorities where we've hidden at least 2 requests before. We can change that number to see what works best. What's your thinking ben?

[benrfairless]

Should we ask everyone who makes a request? I think it's a perfectly valid thing to be proactive rather than reactive?

Ben Fairless Volunteer - Right to Know - @RightToKnowAu

A project of the OpenAustralia Foundation

On 28 Oct. 2016, at 8:36 am, Luke Bacon notifications@github.com wrote:

Does it apply to all requests or just some?

@benrfairless Only requests to authorities where we've hidden at least 2 requests before. We can change that number to see what works best. What's your thinking ben?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[equivalentideas]

This is deployed, see https://www.righttoknow.org.au/new/dibp

I've set GA tracking events on the use of the switch, clicking the email, and submitting the request. We should be able to use this to get a bit of insight into how this impacts usage.

[equivalentideas]

Should we ask everyone who makes a request? I think it's a perfectly valid thing to be proactive rather than reactive?

@benrfairless From a user experiment perspective, adding another form field into the process brings more cognitive overhead (e.g. gives people more to think through, making it harder to get through). So we thought that it's best to not display this unless the person is likely to make a request for personal information—and we're very crudely assessing that by which authority they're requesting from. But this should catch the vast majority people making these requests, based on our existing data https://github.com/openaustralia/righttoknow/issues/584#issuecomment-255305454

Let's see how this goes :) we can then adapt when it's shown to find the sweet spot 🍨

[equivalentideas]

In a week or so we should check the impact of this.

• Has frequency of request hide events changed?
• Has the impacted people making public requests? Less requests? More requests?
• Can we tell anything else by looking at the Google Anaytics tracking I set up

Have we gotten any feedback from people?

Should we keep this?

[equivalentideas]

Should we remove the notice on a bunch of the authorities that we put in the notes? e.g. “Do not request personal information using this site. Why?” https://www.righttoknow.org.au/new/afp

[RichardTaylor]

A related issue is:

https://github.com/mysociety/alaveteli/issues/3542

Which proposes a "report this request" link on outgoing messages which would give public bodies an opportunity to take action in relation to inappropriate requests for, or containing, personal information.

[henare]

We just had a really interesting situation that I think is worth documenting.

Someone emailed us this morning urgently requesting a police report. 8 minutes later they made a request via the site anyway. They almost certainly would have seen this (maybe we can check in Analytics?):

So that means they've then clicked "no" on that form just so it would display the request creation form and then they've filled that in.

This was a scenario we thought of and I think it's interesting it's happened, especially in these circumstances.

[garethrees]

lol, sigh :confused:

It might be worth a followup to the user if you haven't already – might at least give us some idea of their thought process.

Do you have a sense of whether this has made a general improvement yet? The WDTK volunteers were expressing interest in porting it over.

[henare]

It might be worth a followup to the user if you haven't already – might at least give us some idea of their thought process.

Yep, @benrfairless and @equivalentideas have been following up with the requester and the authority - what's been the result of that?

Do you have a sense of whether this has made a general improvement yet? The WDTK volunteers were expressing interest in porting it over.

My gut-feel is that it's made a massive improvement but we still need to run the numbers. Importantly we also need to check if it's resulted in a drop-off of normal requests.

[equivalentideas]

Yep, @benrfairless and @equivalentideas have been following up with the requester and the authority - what's been the result of that?

@benrfairless might have more insight, but as I understand it. The requester said:

<the authority> suggested to visit your site. I'd pay AUD 50 and would receive a copy of the event case I filed with them, back...

@benrfairless then called the authority and they said “general they would refer people to Insurance Services Unit (part of NSWPOL).” Ben then emailed up the GIPA people at the authority, but they seem to have misunderstood the question :S

My guess is that this person Googled GIPA or FOI, came to our site (possibly through an AdWords ad, we can see requests to the authority on that day via google ads), and didn't care that their request would be public. Or was so frustrated (they are extremely frustrated in their emails to us) that they just kept clicking till the request was made and didn't really read the privacy stuff.

[Gemmamysoc]

Hi guys :)

We've seen an increase in WhatDoTheyKnow users writing to us to report invalid requests - most of which are users requesting personal information.

This made me think of this work you've done and that perhaps it's time for us to consider implementing it on WDTK.

What results have you seen from deploying this? Have requests for personal info to the authorities you've implemented this for gone down?

No rush to respond to this! Cheers!

[equivalentideas]

What results have you seen from deploying this? Have requests for personal info to the authorities you've implemented this for gone down?

@Gemmamysoc Good question.

This is why we added the chart upstream https://github.com/mysociety/alaveteli/issues/3545 . Unfortunately we haven't managed to get our Alaveteli to the version with that chart in it 😬 https://github.com/openaustralia/righttoknow/pull/686

The other way to work this out would be to run the calculation code we added for that chart on the Rails console. I'll have a quick look into how feasible that is.

[equivalentideas]

@Gemmamysoc reading this issue in a bit more detail, found the script I made earlier for this kind of this https://github.com/openaustralia/righttoknow/issues/584#issuecomment-255294987

I did a fresh export of the hidden requests following the steps above and used it to make this rough chart in excel:

We deployed the form change on October 28 2016 https://github.com/openaustralia/righttoknow/pull/652 .

It looks like the frequency of requests being hidden has stopped increasing since we implemented this. That seems good 🎉

But why isn't it going down?

An important thing to consider here is that we're also getting an increase in the frequency of requests over the last year. I haven't done anything here to take that into account, but I think for these purposes we're safe to assume that the number of new requests being hidden each week, as a proportion of total new requests each week, has gone down.

We also had two admins for a large part of the last 10 months, which you'd think would mean more requests that should be hidden would be getting hidden.

Thoughts anyone? I'm not great at statistics so don't be shy about questioning my logic here :)

[equivalentideas]

Some more data for you @Gemmamysoc 📈

Since we deployed the interface on Oct 28, we've been tracking click events on the "yes" and "no" form options and on the email address of the authority that we present to people who want to make a request for personal information.

I guess all that tells you is that people have actually been using the interface.

This chart is just clicks on the email address per week:

Dunno how useful that is, but hope it helps @Gemmamysoc 🍹

[Gemmamysoc]

Thanks loads for getting back to me on this @equivalentideas! One of our new WDTK volunteers recently suggested implementing a interstitial page before request submission like this, so it was really nice to be able to point him towards your existing work on this, and the news that it looks like it's been helping :)

I think it may be a while before we implement this on WDTK, but at least we know it's available when the dev team get a chance!

Cheers again!

[equivalentideas]

@Gemmamysoc my pleasure. It's good to check in on how this is going anyway 👍