GDPR related changes

[mmd-osm]

See https://wiki.openstreetmap.org/wiki/GDPR/Affected_Services

Some API calls are also executed through CGImap and need to be restricted there as well.

cc: @simonpoole, @woodpeck : please feel free to add further details about agreed upon scope and relevant timelines.

[zerebubuth]

I don't think the suggested changes are necessary. The LWG position paper sets out a case in which the possibility that a user is personally identifiable from their collected edits requires that we remove all user information from all non-authenticated responses. This proposal suffers from two flaws:

1. The vast majority of users are not personally identifiable from their edits.
2. Anyone can sign up for an account and use an authenticated API call to get the user information.

In cases where the user might be identifiable from their edits, then we should look at remedies for that. Which might mean extra tooling and analysis to find out what common mistakes are being made (e.g: not enough edits to mask the user's home location, calling things "my house", etc...). However, mandating these changes to the non-authenticated API would appear to be simultaneously very disruptive and ineffective at protecting users' information.

[simonpoole]

@zerebubuth I believe you are misreading something there. The GDPR only requires that data is associated with a "identifiable"(1) person, not "personally identifiable" (I assume you mean identifiable by personal name by that) to be considered "personal data" Note the GDPR clarifies that even pseudo-anonymized personal data, which you could argue is the case with OSM, is considered personal data.

The exploratory part of the 2nd section is simply to show that it is possible to associate edits with specific individuals even without uid and display name, and illustrate that it would be very difficult to arrive at a 100% solution. That doesn't have any bearing on removing the basic linking information (uid etc), just on where we draw the line.

The proposed solution with removing the information from public APIs, is a compromise, not a perfect solution. It allows us control at least to a degree what is happening and convey the information to the API and website users what uses of the API are considered problematic from a data protection point of view.

1. the requirement is simply that the information is associated with a specific (different word for "identifiable") person, which is in general the case with an OSM account.

[zerebubuth]

The GDPR only requires that data is associated with a "identifiable" person, not "personally identifiable"

A particular natural person is not identifiable from their OSM UID, as it's just a meaningless machine-assigned number. Whether they are identifiable by their display name is entirely within their own control and subject entirely to their own consent. The existence of an account ID or display name does not imply an identifiable person, and assuming that would mean that all public online contributions would have to be anonymous, for example in comments, forums or blogs. Some social media sites even encourage the use of identifiable real names and (sadly) show no signs of being shut down by GDPR.

Therefore, the only problem is when a person might be personally and individually identified by the sum of data that they have contributed under a single account. While possible, it seems likely very rare, and something that we can handle reactively rather than by a very disruptive and ineffective API change.

[mmd-osm]

From what I've experienced so far, different development teams are super eager to try out a working API with limitations in place as outlined in Appendix B, and work on identifying and fixing any issues.

I'm a bit disputing that this change is "very disruptive". Here's why. While I didn't see any major issues with iD so far, one issue I raised with JOSM was fixed in less than an hour after reporting it. :+1: Potlatch 1 & 2 could be a bit challenging from a technical point of view, but that's just a gut feeling.

I don't want to get involved too much into a "meta" discussion in this issue, i.e. if the proposed measures are at all useful or not. I trust LWG folks to have put sufficient thought into the overall topic to come up with a somewhat reasonable approach. That's not to dismiss your comments, I rather believe https://wiki.openstreetmap.org/wiki/Talk:GDPR would be a better place for such a discussion.

[don-vip]

Any news from this?

[mmd-osm]

Some update in the most recent OSMF board meeting minutes: https://wiki.osmfoundation.org/w/index.php?title=Board/Minutes/2018-08-16#Request_for_proposals_related_to_API_changes

Quite frankly, I'm not sure what this all means. There must have been some more discussion in the meeting which isn't covered in the minutes.

Doing those changes would be almost trivial for cgimap (except for fixing all those test cases, see code snippet above), if there only was consensus as to what should be excluded from the data.

[mmd-osm]

https://wiki.osmfoundation.org/wiki/Board/Minutes/2018-09-20#API has another update. Looks very similar to the previous one, except that they dropped the "(paid position)" suggestion.

[simonpoole]

@mmd-osm my understanding is that the "paid" part is implicit in the RFP process.

[tomhughes]

Why are they faffing about with an RFP without even approaching the relevant development teams first and asking if they are willing or able to do it?

[woodpeck]

What would be the relevant development team of cgimap? Are there any members of that team not participating in this dicussion?

[woodpeck]

@mmd-osm, what further work would be required on your prototype implementation to use it productively, assuming that the requirements were exactly as outlined on the wiki page?

[mmd-osm]

Are there any members of that team not participating in this dicussion?

At least @pnorman isn't mentioned yet...

what further work would be required on your prototype implementation to use it productively, assuming that the requirements were exactly as outlined on the wiki page?

Well, the prototype simply covers the most basic aspect of hiding a few fields in XML/JSON response for non-authenticated users, and currently only supports OAuth.

There's still plenty of work ahead:

Authentication

First of all, the wiki page isn't quite clear on required authentication mechanisms. While the rails port does support Basic Auth, the respective implementation for cgimap isn't merged into the master branch yet.

For compatibility reasons, I'd say we need Basic auth as well here.

Test cases

Then there's the issue of test cases: adjusting them one by one seems quite tedious, as it would require some kind of split up into a non-authenticated and authenticated case.

Maybe this could be done in one central location like check_content_body_xml (test_core.cpp), where relevant fields can be automatically dropped from the "expected" xml.

Still, this would require setting up dedicated user accounts as part of the test cases where this isn't the case yet as of today. Then run the each test case twice, once with and without authenticated users.

Rejecting calls for unauthenticated users

/api/0.6/changeset/:id should be disallowed for unauthenticated users. This isn't part of the prototype yet.

[tomhughes]

Basic auth should be going away, not getting extended!

[mmd-osm]

Basic auth should be going away, not getting extended!

I'm with @pnorman here: https://github.com/zerebubuth/openstreetmap-cgimap/issues/145#issuecomment-392383351

If we later decide to drop it, it will be more or less a one liner change.

By the way, basic auth is already deployed on https://upload.apis.dev.openstreetmap.org/ for testing, just in case you were wondering.

[don-vip]

Latest news: https://blog.openstreetmap.org/2018/11/03/rfq-changes_to_rails_api_and_cgimap/

I'm surprised by the statement about payment. So the OSM API is no longer maintained by benevols? Sad.

[simonpoole]

@don-vip I think that is a bit of a misconception, it is simply that nobody has shown an interest in voluntarily making the changes and it is neither the first not the last time that there has been paid for work on the API (either directly or by working on a friendly employers time).

[mmd-osm]

I’m closing this issue for the time being as the topic has disappeared from recent osmf board meeting minutes and I don’t know who’s currently in charge with coordinating and driving this topic forward.

[woodpeck]

I don't think it should be closed just because we haven't yet found anybody to work on it. It remains an important issue and the fact that the board has not treated it with the highest priority doesn't mean we should stop worrying ;) the board is still looking for someone to implement the required changes. Recent developments regarding moving message constructions from controllers to views should make this task a little easier (thank you).

[zerebubuth]

It remains an important issue ... the board is still looking for someone to implement the required changes.

I really don't think there are any required changes for GDPR. As I've explained at length, hiding the display name and UID from non-logged-in users is simultaneously very disruptive and totally ineffective at protecting users' information.

Further, it has become clear that not a single other site with social features is doing or planning anything comparable under the guise of GDPR - and they remain active without being sued into oblivion. Therefore, I think it's untenable to pretend that this is required of us by GDPR.