monty241
commented
4 years ago Reading attack scenario 1, I've read:
However, such access permissions represent a very low hurdle to the attacker: by running a government approved app on a rooted phone equipped with the right hacking tools, the access credentials, or the data itself, can be extracted.
In my humble opinion this is the same approach as that hinders the OAuth2 code grant flow being used on untrusted devices while protecting application secrets. Sadly enough even industry leaders now ship products with necessary application credentials packaged, by lack of a good solution.
With OAuth a man-in-the-middle attack is easy in general, since the HTTPS traffic in general can be accessed since the device is untrusted and can be manipulated such as by re-routing traffic and/or loading certificates. Risks on attack scenario 1 can be mitigated but not removed by adding additional layers of "undocumented magic" in the client's algorithm or seeded data such as a client certificate. Changing the algorithm in an undocumented way is not really desirable.
Forwarding data from the device untrusted for the application's purpose to a central infrastructure under control of the application's owner (or a delegated party to reduce control by the state) indeed seems like a solution that is able to secure privacy in the long term and reduces the number of actors that can harm privacy. Educating millions of possible malicious users and their devices to meet software supplier expectations has so far proven impossible to achieve, especially in the Android eco-system. A central infrastructure allows for easier auditing and change when security issues arise.
kennypaterson
kholtman
lciti
kdagley
gardners
vaudenay
tom-leclerc
ramsestom
burdges
lbarman
christiano-git
pzboyz
IcyApril
Introduction
This issue relates to the privacy risk analysis done in the April 12 version of the white paper. Basically, I believe this analysis grossly under-estimates the risk that infected users will be de-anonymized. I will illustrate this with some attack scenarios below. I will also outline a redesign approach which I believe will better meet the goals of the project.
These comments are mainly informed by my experience in doing industrial R&D work on the privacy issues for various types of smart phone based location tracking infrastructure. Based on my experience with accuracy issues, I have other concerns too, but these are mostly reflected in the issue reports of others already, so they are not included in the long write-up that follows.
Threat considered
The threat model and analysis (section 5.1 of white paper dated April 12) recognizes the risk of an eavesdropper who uses a stationary or mobile device that collects more data than just broadcast EphID data. This device might also collect and store camera images, or the plain GPS location of the eavesdropping device and point in time that each specific EphID was captured. As soon as the eavesdropper has access to (information allowing the matching of) the EphID values of self-reported infected user, this eavesdropper might be able to use the extra collected data to de-anonymize one or more infected users. Success rates can be very high here, especially if a lot of 'crowd-sourced' eavesdropping information is combined.
The threat model analysis made in the white paper estimates that an eavesdropping attack is costly if a large area has to be covered, and also notes that it is illegal under EU law, leading to the conclusion that the privacy risk associated with this attack is low. I will argue that the analysis made is incomplete. Specifically, the per-victim up-front cost of implementing certain types of scalable, geographically widespread eavesdropping and de-anonymization attacks is low. Though these attacks are typically illegal under the GDPR and/or other laws, this in itself is not enough to conclude the attacks will not happen. What matters is the equation that compares the immediate benefits to the attacker(s) against their risk of getting caught and being brought to justice. This couples to the ability of EU governments to effectively enforce the GDPR in the worldwide app economy and advertising/tracking ecosystem, and this ability is unfortunately very low.
I will now outline two specific attack scenarios that I have not seen mentioned earlier in other issue reports.
Attack scenario 1
This attack is performed by an independent app author and the users of that app. Imagine that the government-approved or government-promoted standard app for a county will tell its uninfected users whether they have recently encountered an infected user with high probability, but for privacy reasons it does not reveal where and when this encounter might have happened. Now consider an app author who releases an improved tracking app that will also tell its uninfected users exactly where and when the encounter took place. This will often have the effect of de-anonymizing the infected user. In risk terms, it exposes infected users to a much much greater risk of de-anonymization than was intended or promised, so it will suppress the willingness of these users to self-report. Basically we get a bad dynamic here, that goes beyond mere privacy concerns.
To make this improved but attacking app work, the app author needs to design an app that a) scans for and logs EphID beacons in just the same way that the approved app does, b) logs not just the EphID values collected, but also the time and place at which each value was collected, and c) has access to infected user EphID values, or a means to match them with the logged data. This matching means can be implemented by a routine that makes a query call to a remote server set up by the app author, a server located outside of the EU, with this server containing the necessary match-enabling data related to infected users. Now, one mitigation against the creation of such a server, as envisaged in the white paper I believe, is to implement access permission restrictions to the official database(s) that hold this data, so that only legitimate government-blessed apps can get the data. However, such access permissions represent a very low hurdle to the attacker: by running a government approved app on a rooted phone equipped with the right hacking tools, the access credentials, or the data itself, can be extracted. The necessary expertise can be hired on the dark-nets I believe, and the cost per affected user will unfortunately be very low.
As another mitigating approach, governments could make an attempt to discourage such app authors by prosecuting them, but if they reside outside of Europe, and if matching data processing is done outside of Europe with some plausible levels of deniability built in, this will all get gets very difficult. Banning the apps from the official app store might be possible, but app store policing is an imperfect game, and on Android there is the option of side-loading. Going after end users who install the improved app will also be difficult: there are privacy and right-to-repair and right-to-root implications. At a political level, attempts to ban these improved apps and their use might be hugely unpopular with the users of the improved apps. Such users might organize, or be organized, into pressure groups that force the government to break its promises to infected users, accepting a mission creep that gradually erodes their privacy.
Note that the attack scenario outlined in GitHub issue #37 can be seen as a particular refinement of the above scenario. Definitely, the mitigation considered under design 2 of the white paper can reduce the risks of de-anonymization: in my view it helps but it is not sufficient to eliminate the problem.
In terms of lowering the risk to acceptable levels for infected users, the most effective mitigating action I can see see is to change the data flow design, so that the government app can no longer download any descriptive information about infected user EphID beacon values. This means that matching will no longer happen on uninfected user's phones. Instead, the government approved app will have to send the EphID beacon values collected by the uninfected user's phone to a trusted third party database server or matching service, where they will be matched with the infected user EphID beacon values, with only a only a single yes/no bit, or some slightly richer result value, being returned to the phone of the non-infected user.
To make the above mitigation measure effective, a query rate-limiting system with active monitoring is needed on the above third-party service. This is needed to lower the risk of success of attacks which use series of specially constructed queries to narrow down time windows, or attacks with queries designed to extract all latent information in the database. Beyond a rate limiting system, a proof-of-right-to-query approach, based on fuzzy matching to tracking logs collected by the infected person's app, has great promise as an effective countermeasure. There are more options, I won't expand further here.
To be clear, the above mitigating solution definitely increases the privacy risks for non-infected users: their phones will now be sending some data to a third party, data that is never sent in the current design. However, I believe this solution direction is needed to achieve acceptable levels of privacy risk for infected users.
In fact, the current design can be said to be unbalanced in exactly the wrong way: it assigns all privacy risks to infected users, just so that the non-infected users do not have to rely on trusting some third-party data matching service operator. However, non-infected users still have to trust other third parties: e,g. smart phone OS makers, and the makers of the government-promoted app, so I believe that the burden of requiring trust in an extra third party server or service is not a big step.
Attack scenario 2
This is an attack by an ad library developer. In today's app ecosystem, many free apps rely on revenue from showing ads, and these adds are typically inserted using third party ad libraries that are linked into the free app. These libraries typically collect privacy-sensitive data from the phone. There is a legitimate need to collect this data and send it to the ad library maker back-end systems: it can significantly improve the selection of the ad to show. But this also gives the ad library maker the technical ability to go beyond acceptable or allowed data processing or sharing practices. The ad library back-end might process or forward collected data in a way that is illegal in the jurisdiction of the app end user, illegal even if the app end user has pressed 'accept' on some multi-page agreement. This is relevant here because ad libraries can also include code that accesses the GPS and Bluetooth systems of the phone (see also the links further below). Typically, access requires that the user also clicks OK on a permission dialog presented by the smart phone OS, but most end users have been trained by industry to click OK always. We may have some specific luck here: for power saving reasons, Bluetooth is often turned off and this is somewhat decoupled from the permissions system. To outline the attack data flow: GPS and Bluetooth EphID data is collected by the ad library, and sent to the ad library developer back-end systems. The ad library developer then sells this data to diverse data brokers, typically sharing a revenue cut with the app developer who included the library. When sufficient EphID and related sensor metadata is present in the databases of these of data brokers, and if these brokers also obtain the matching data for infected users, these brokers will be able to de-anonymize many infected users with high confidence. If past history is any guide, some of them will not hesitate to sell both high-confidence and low-confidence identifications. So this is a scalable, low-cost, and profitable attack that will typically proceed without the smart phone users even being aware that their phone is helping in the attack, and that they might even be running some legal risks themselves because of this.
For some background information on the above, see for example https://www.ftc.gov/news-events/blogs/business-blog/2016/09/ad-libraries-app-developers-check-out-advice and https://money.cnn.com/2013/12/18/pf/data-broker-lists/ To be clear, the last time I studied the state of the smart phone add library and data broker ecosystem in detail was around 2016. At that time it was a privacy and privacy enforcement dumpster fire. I have not checked if and how much it might have improved since. My expectation is that attack scenario 2 is still feasible right now, especially in the Android ecosystem. I would be delighted if someone makes an analysis which concludes that scenario 2 is now low-risk.
Mitigation: In theory, a perfectly enforced GDPR would make the above scenario impossible. A problem here is that an ad library developer may hide behind a claim that they and all their partners will never process or forward data in a way that violates the GDPR or associated safe harbor rules. This makes enforcement very difficult. Historically speaking governments have done very little in terms of trying to develop an effective enforcement capability. The only effective mitigation that I see is the third-party service based matching I described earlier.
Concluding remarks
Reading the project white paper and other documents, there are many things to like. I like especially that the design and analysis recognize the role which the GDPR and GDPR enforcement can play in improving privacy outcomes. However, I believe the design and risk analysis are currently very unbalanced.
The expected societal result of this unbalance, if not corrected, is that infected users will be asked to take a fairly large risk that they will be de-anonymized. This is especially bad because the consequences to infected users of being de-anonymized can be very severe, compared to the consequences for non-infected users. I believe it is better to pivot the design, to create a more equitable distribution of risks and benefits among user types.
Once we accept the idea that a trusted third party should be in charge of matching, that it cannot happen on phones themselves, a large design space opens up that allows risks to be managed and re-balanced differently. In one corner of this space, it is the uninfected EphID related data that is stored for a few weeks outside of the originating phones, whereas infected EphID data is read out once and then immediately discarded once matches have been made.
Note that all of the above does not apply to the share-with-epidemiologists part of the proposal. This has a different implied risk structure, which is less problematic in my view. In other issue descriptions, I have seen suggestions that the project should more strongly decouple this part of the design. I support these suggestions.
I believe the project is partly motivated by a very valid trust-in-government question. However, there are several trust-in-government questions that need to be asked. To put it bluntly, regarding the threat model of eavesdropping and associated data processing, I have a fairly low trust in the current ability of EU governments and institutions to enforce GDPR adherence in the various private party players and commercial ecosystems described above, especially when these players reside outside of the EU. I definitely hope that GDPR enforcement abilities will improve long-term, but I see no quick fixes here. By comparison. my trust in an EU government's ability to set up, and keep honest, a third party service that will handle data from both infected and uninfected users is much higher.