Distance calculations - accuracy attained vs. accuracy required

[adarrel753]

Use case scenario:

• Bob and Alice are less than 2 meters apart separated by a wall
• The received bluetooth signal strength is less on both phones than it would be in the open air.
• Neither person can cough/sneeze through a wall

Q. Can it be presumed that when environmental factors reduce received signal strength they always reduce risk of cross-infection too or is further detail required?

[coalacorey]

No, that can't be presumed in general, but it suffices for our use cases. If Bob sits in a Faraday cage (or microwave) Alice phone won't receive his signals while they still are able to infect each other. While that use case is ridiculous the received signal strength and risk of cross-infection does not always correlate in reality. BLE is not used because it is the optimal solution to the problem but because it is what he have and still a better indicator than others.

[adarrel753]

Fair enough, that makes sense. Another scenario that is less ridiculous would be this:

• Bob visits Alice (his mother) who he knows is sick with Covid-19
• They commuicate through a pane of glass (i.e. a kitchen window)

It would seem to me that the system could produce false positives from the above scenario. If Bob knows this about the system, he could simply switch the app off during that visit to help avoid them being produced and causing unnecessary alarm to others.

I know that a perfect system cannot be built and was thinking more about whether the general public could be told how/when to use the app and how/when not to.

[doug-leith]

These sorts of concerns are also backed up by recent measurements, see https://www.scss.tcd.ie/Doug.Leith/pubs/bluetooth_rssi_study.pdf

[Xenoamor]

Thanks for the paper @doug-leith, an interesting read.

I think false positives are an inevitability but the government will have the option of "turning the dial" on the criteria required to generate an alert. These could be done on minimum RSSI requirements or contact time and could even scale differently in different areas. This in effect would generate a variable lockdown that can be used to control the rate of infection to ensure nearby hospitals have enough beds.

There are economic reasons why they might turn that dial but unfortunately we will never know if that is the case. There's likely to be psychological factors as well, too many lockdowns and app participation may reduce as people switch it off.

It is worth noting that they'll have billions of data points and perhaps from such a large sample size there will be something meaningful come from the BLE data but it would definitely be something more woolly than distance.

I don't think you could have a system that works per individual reliably using BLE alone but you'll have something "good enough" for a large populous, especially when combined with social distancing and testing.

In app information as you suggest @adarrel753 could definitely help. I believe there's already a section that tells NHS workers to turn it off when at work already

[doug-leith]

A couple of brief comments in reply:

• Its not just false positives that are a concern, but also false negatives. Those measurements I gave a link to suggest both might be quite likely in real environments, certainly if deciding upon proximity by simply thresholding on observed RSSI.
• To the best of my knowledge there are no public results at all on the error rate of the NHS app at detecting proximity (or any other app for that matter). Perhaps there is private data, in which case it would be good to make it public. But if not then there seems little basis for assuming that it will be "good enough", and the measurements which are available (that link again) do suggest reasonable cause for concern.

[mcb30]

To answer the question raised in #16 as to how contact tracing can work at all given that RSSI is essentially meaningless: It has to be based on a combination of "signal strength is high enough to make contact at all" combined with other data such as "duration of contact" and optionally "location of contact". In many cases, there is no viable way for an automated system to acquire all of the relevant data, and an assessment of risk will require knowledge held only by the end user.

For example: suppose that I am using a fully decentralised model. My phone sees an alert for a particular contact ID, and can tell from my phone's own logs that this contact took place from 4:17pm yesterday and lasted for 30 seconds. My phone dismisses the alert as being below the risk threshold, and does not raise a notification to me.

Suppose now that the contact lasted for 30 minutes. My phone assesses that this is above the risk threshold for alerting me, and raises a notification about the potentially hazardous contact. I know that at this time I was sitting outside in a park and that nobody came close enough to be of concern. I therefore dismiss the alert.

Suppose that I am walking outside and am passed by someone who is coughing violently and failing to keep a safe distance. I press a button to tell my phone to treat even short-duration contacts around this time as being high risk. My phone subsequently sees an alert for a contact with a duration of 30 seconds at this high-risk time, and so notifies me.

As this hopefully shows: there is a lot of information that feeds in to the risk calculation, and Bluetooth signal strength is only a very small part of this.

[doug-leith]

Thanks, it does seem likely that more than RSSI will be needed. You mention "risk calculation", but at the moment there seems to be little data to inform such calculations. Doesn't that mean its more like guesswork and hoping for the best? Surely these sorts of considerations point to the need for some concrete actions re the NHS app so that decisions might be made in a more informed way, including: (i) Carry out tests to evaluate the impact of such additional social techniques on the contact tracing error rate. And as a starting point, carry out tests to clarify what the baseline error rate is without such extra measures.
(ii) Adoption of the sorts of social technique you mention suggest that changing the app to provide some support for these might be v helpful e.g. to allow people to easily note when they are alone and not at risk.
(iii) Clarify what level of proximity detection error can be tolerated while still achieving infection control.

[edent]

Hi all, some really good discussion here. The details of the algorithm are published at https://faq.covid19.nhs.uk/Risk-scoring%20Algorithm%20-%20For%20Publication.pdf

Part of the Beta on the Isle of Wight is to validate how well BLE distance estimation works in practice. Similarly, the RAF test also looked at whether it was feasible. We are investigating the RF properties of a huge number of chipsets and phones to understand how RSSI is effected.

We're going to close this issue, because it doesn't relate to a bug with the app. You are welcome to keep discussing it on this thread.

[sftcd]

Hi (I worked with Doug on the report linked above). I think it may be an error to close off this issue. While it may or may not be a client side bug, ISTM that the distance estimation errors will still feed into false positives/negatives. While it's true that uploading the RSSI/TxPower and contact timing does provide more information I'm not clear that the RSSI->distance mapping will be any more accurate just because one applied a function after a distance is estimated. The contact duration does provide more information though but I'm not clear how very noisy RSSI values can be handled well even with that TBH. I also note the description of the RAF tests didn't make clear whether or not that was in an environment that's tricky for radios e.g. like the train carriage in which we tested, or when sitting on a metal park bench.

[skenaja]

I'm not sure how many traffic jams there are on the Isle of Wight, but if these photos are anything to go by from today's North Circular in London, I worked it out that just 30 mins of nose-to-tail multi-lane stopped traffic is enough to trigger a chain of potential false positive contacts for 2 cars side-by-side (I assumed 4m driver separation, using the risk-scoring algorithm), and 90 mins for cars in front or behind (assumed 7m separation).

Those times might be easy to accumulate in 10 days of commuting on a busy route, or when queuing up for drive-through Covid test sites, so the app's advice for turning off bluetooth might be worth extending to solo drivers.

[sftcd]

On 13/05/2020 12:28, skenaja wrote:

I'm not sure how many traffic jams there are on the Isle of Wight, but if these photos are anything to go by from today's North Circular in London, I worked it out that just 30 mins of nose-to-tail multi-lane stopped traffic is enough to trigger a chain of potential false positive contacts for 2 cars side-by-side (I assumed 4m driver separation, using the risk-scoring algorithm), and 90 mins for cars in front or behind (assumed 7m separation).

I have it on my todo to check out RSSI between two adjacent cars. If someone's already done that be great to know what they saw.

Those times might be easy to accumulate in 10 days of commuting on a busy route, or when queuing up for drive-through Covid test sites, so the app's advice for turning off bluetooth might be worth extending to solo drivers.

That's where the google/apple API may have an advantage. My guess is they will have access to other sensors and the underlying knowledge that might help with some such scenarios. But, doing that well would still be pretty hard I guess.

And of course that only helps with false positives. I don't know how one might tackle false negatives caused by highly noisy radio environments.

Cheers, S.

[skenaja]

Hi all, some really good discussion here. The details of the algorithm are published at https://faq.covid19.nhs.uk/Risk-scoring%20Algorithm%20-%20For%20Publication.pdf

Looks like this URL has changed or the doc has been withdrawn - now giving a 404 error :-(

[adarrel753]

Sometimes I think asking for real-time user input would be helpful. If Bob switched a 'Protective Mode' on during the visit to Alice the false negative for that contact event could be avoided very easily, along with the cascading events afterwards, I.e. the people Bob then walks by on the way home, in the supermarket etc. NHS staff could use the same button when wearing PPE gear

[Xenoamor]

@skenaja looks to be here now: https://nhsbsa-socialtracking.powerappsportals.com/Risk-scoring%20Algorithm.pdf