[1.2] "Passive measurements use surveillance..." -- the Oxford definition of surveillance is "[noun] close observation, especially of a suspected spy or criminal."
[1.3] encapsulates all measurements as being sources of attack. There are 12 explicit definitions, 9 of which start by "An attack".
It's important to recognise that a draft like this would be beneficial to a wide audience. At the same time I am unable to see how an outside observer or reviewer, e.g. institutional review, using guidelines of this form (and likely have to consider liability along with safe and responsible practice), wouldn't automatically be forced into a starting position of defensiveness against measurement. (Aside: one additional suggestion may be to use 'responsible' alongside 'safe', which is a common convention in the research community.)
The 'risk' in this current draft is that its current set-up reverses a lot of the incredibly hard and tireless efforts from the measurement community to define, account for, report, and review best practices, as well as call out violations.
Second, there are a few subtleties missing from some 'Guidelines' that cannot be overlooked. Their omission could lull a reader into false senses of security that they've checked certain boxes. In particular,
[2.1] suggests that informed consent is sufficient, when in fact we know that even fully informed consent in some circumstances may not be enough. For example, the value and merit of consent to statements like "by helping this measurement you could go to jail" is questionable, at best. Alongside, the seeker of consent has to be taken into consideration; for example, there is no way to know the effect of a consent-seeker on a consent-giver who may be more or less inclined to trust a multinational corporation, foreign government, civil liberty group, or a university lab -- these are just not the same, and an individual may trust more or less depending.
[2.3] This is a good one! If it's helpful, even the seemingly obviously 'ok' can have caveats. Speedtests are a great example. It might be reasonable to assume that any speedtest server is happy to receive a test request, and any subscriber who pays for access is entitled to request the test -- but what about the transit provider in-between?
[2.4] Also great, and easy to overlook. Here, too, however, I wonder if all requests are equal? As an example, what does it mean for Google to request do-not-scan? (I honestly don't know!) One possible addition may be to suggest that "self-identify" is good practice, especially when releasing measurement tools into the world. For example, the popular zmap tool self identifies itself by placing "54321" in the IP-ID field.
One last suggestion: It's hard to know which are the most useful references in this space, so a draft like this would be a great place to have a comprehensive list of pointers to best and current practice in this space (e.g. the CACM written by Partridge and Allman), and docs that might include case studies. Even better might be to have one or two sentences for each to describe why the reference is useful, or what it offers. (Acknowledging in advance that distilling down from multiple inputs is a challenge in itself, and would be a great feat!)
[1.2] "Passive measurements use surveillance..." -- the Oxford definition of surveillance is "[noun] close observation, especially of a suspected spy or criminal."
It's important to recognise that a draft like this would be beneficial to a wide audience. At the same time I am unable to see how an outside observer or reviewer, e.g. institutional review, using guidelines of this form (and likely have to consider liability along with safe and responsible practice), wouldn't automatically be forced into a starting position of defensiveness against measurement. (Aside: one additional suggestion may be to use 'responsible' alongside 'safe', which is a common convention in the research community.)
The 'risk' in this current draft is that its current set-up reverses a lot of the incredibly hard and tireless efforts from the measurement community to define, account for, report, and review best practices, as well as call out violations.
Second, there are a few subtleties missing from some 'Guidelines' that cannot be overlooked. Their omission could lull a reader into false senses of security that they've checked certain boxes. In particular,
[2.1] suggests that informed consent is sufficient, when in fact we know that even fully informed consent in some circumstances may not be enough. For example, the value and merit of consent to statements like "by helping this measurement you could go to jail" is questionable, at best. Alongside, the seeker of consent has to be taken into consideration; for example, there is no way to know the effect of a consent-seeker on a consent-giver who may be more or less inclined to trust a multinational corporation, foreign government, civil liberty group, or a university lab -- these are just not the same, and an individual may trust more or less depending.
[2.3] This is a good one! If it's helpful, even the seemingly obviously 'ok' can have caveats. Speedtests are a great example. It might be reasonable to assume that any speedtest server is happy to receive a test request, and any subscriber who pays for access is entitled to request the test -- but what about the transit provider in-between?
[2.4] Also great, and easy to overlook. Here, too, however, I wonder if all requests are equal? As an example, what does it mean for Google to request do-not-scan? (I honestly don't know!) One possible addition may be to suggest that "self-identify" is good practice, especially when releasing measurement tools into the world. For example, the popular zmap tool self identifies itself by placing "54321" in the IP-ID field.
One last suggestion: It's hard to know which are the most useful references in this space, so a draft like this would be a great place to have a comprehensive list of pointers to best and current practice in this space (e.g. the CACM written by Partridge and Allman), and docs that might include case studies. Even better might be to have one or two sentences for each to describe why the reference is useful, or what it offers. (Acknowledging in advance that distilling down from multiple inputs is a challenge in itself, and would be a great feat!)