Suggestion for better diagnostic of "Contact"

[frarude]

The bluetooth signal alone is no good indicator for "being in contact" with somebody. (As I mentioned earlier).

My suggestion would be to combine it with an ultrasonic audio beacon. If you are close enough to get a "bluetooth contact alarm" you could send out an ultrasonic beacon to see if the devices are really in the same room. This could work through fabric but not through walls and would reduce false positives.

The technology already exists , see https://www.sec.tu-bs.de/pubs/2016-batmobile.pdf

[lbarman]

Hi @frarude, thanks for the suggestion. Merging also @leenaars comments from https://github.com/DP-3T/documents/issues/188: Use of near-ultrasound in addition to Bluetooth could be helpful to reduce the amount of false positives. See: https://nlnet.nl/project/Simmel or the Austrian Red Cross app.

One reason we didn't consider sound is for obvious privacy reasons (recording through the microphone); what is your opinion on this ? do we know if in these projects the audio stream is treated purely locally ?

[frarude]

The decoding (and sending) of ultrasonic audio beacons can be done completely locally. But since it costs some computing power it should be done after bt detection, to ensure the result; not permanently. See the simple algorithm on page 14 of the mentioned article which was found in the wild and reverse engineered by our researchers.

[snakehand]

I would be happy to contribute with signal processing code - I have experience with SDR / IQ processing from both ham radio (low bandwidth communication) , and other areas.

[leenaars]

In a physical device like the Simmel, near-ultrasound processing could be dealt with in a completely trustworthy way by combining frequency filters on the one hand and fully local processing on the other. That way the incoming acoustic signal should be cleaned of anything that is outside the frequency band of interest. The device is also offline by design, and has no capacity to store recorded audio anyway. Arguably, together with real ultrasound such a solution would offer the highest privacy guarantees. @frarude: I would not per se rule out having the audio processing done independently or in parallel to BTE handling. In an urban environment, there would always be so many devices "near" that you automatically end up with continuous scanning anyway.

In a mobile phone, there are far less guarantees and control points. If you can say "hey Google", or "Hi Siri", and the device responds, that means there is already something listening 24/7 in on someones microphone. If you turn those features off in the OS, then (assuming you trust the vendor) it will depend on the app how it subsequently handles matters - no blanket statements to be made about that. The same holds for BLE, right? So establishing whether or not audio processing is dealt with locally is done on a case by case basis, by inspecting the actual implementations.

What the OS or the hardware does in the background, is probably neither transparent nor controllable. So if users do not trust those solutions, moving to an open hardware device that is verifiable from top to bottom and has physical constraints to only do the essential task of contact scanning without any risk of spillover would be the better option whatever the sensing technique ...

[gardners]

Leenaars is totally correct here:

First, it is possible to have a device whose intended function is to monitor audio, and is not online, and does not have the capacity to record voice, and whose hardware and software design is open and verifyable in these regards.

Second, phones are currently such a sesspit of privacy problems, that doing this on a mobile phone is really not adding significantly. Again, Google and Apple could offer to solve this, by creating a microphone API.

Third, it would be possible to make a tiny USB plug-in that had a dedicated microphone and data collection on it, so you don't have to carry around a 2nd device, or have your mobile phone microphone on. It just uses your phone as the power source for convenience.

Fourth, the near-ultra-sound approach of using an off-the-shelf MEMS microphone and small speaker has the potential to be very cheap, and due to the poor efficiency of cheap microphones in the ultra-sound range (and that of the MEMS microphones), this will intrinsically limit the range in a very useful way.

In short, this has a lot of advantages for privacy, that the whole bluetooth / fully-integrated-into-insecure-smart-phone approach can't offer.

In the very least, the DP-3T system should be made with a layered architecture where the format of data, and the data collection sensor layer are both separate and separate from the processing algorithms. The coding overhead for this is very small, and will likely actually save time, even on the current BT only approach, by making it easier for multiple teams to work in parallel on the different aspects. In short: Have clearly defined interfaces.

@Leenaars: Do you think that it would be worth doing some tests with near-ultrasound on some existing hardware we have here (MEGAphone prototype boards with MEMS microphones and speakers).

[snakehand]

Just as a point of reference: This company https://forkbeardtech.com intends to use bluetooth together with the built in microphone and audio in the 20kHz region for indoor positioning. I have seen demos of this, and it works surprisingly well.

Another thought is to use simple phase shift keying such as for instance used in https://en.wikipedia.org/wiki/PSK31. If for instance chirps are spread out with 50 Hz spacing over 1 kHz, then a high degree of discrimination can be obtained by running FFTs and comparing phase over time. This would be similar to how OFDM signals are demodulated.

[ineiti]

Chiming in a bit late - I also think this is a very useful extension ;) There is a paper here https://asa.scitation.org/doi/pdf/10.1121/1.4860156 that did successful ping measurements of smartphones, and it seems to work quite nicely. I haven't yet completely understood how the paper gets rid of the delay in the transmission pipeline, but it seems to work.

But talking with friends, they already hesitate to install an app that does bluetooth locally only. So I guess they really would be against having the microphone on all the time. And no, they don't use Siri or "OK Google"...

[leenaars]

@ineiti: the hardware variant that started the thread would be acceptable to your friends I believe, because it should conceptually be seen more as an audio sensor than as a microphone that is always on. I am personally not too convinced of the privacy guarantees of other solutions.

[ineiti]

@leenaars - OK - so I missed the hardware version ;) I was thinking about the standard microphone. And there would be a lot of people not wanting to have that, of course.

Only nerds like me who trust the source-code would be OK with it. All the others just use Siri and "OK Google". But not a sonar-ping of an app that can actually help them...

Anyway, for the creator of the sonar-ping, this would of course be a good way to make huge amounts of money ;)

[gardners]

The transmission pipeline latency subtraction could be done by listening to the microphone and measuring the apparent latency of a "0 metre" distance, and then subtracting that from the ping time of more distant contacts.

[oetiker]

As I user I would be glad to enable the audio option as it would provide me we more accurate tracing ... Especially when moving in a busy environment as the number of false positives could be greatly reduced. I think this benefit would outweigh the perceived potential for abuse for many users. If the audio receive option was user-configurable then those who are uncomfortable with that aspect of the functionality would be free to disable it.

[ineiti]

@oetiker - me too I would enable the mic if it helps me getting less false positives and if I can see the code. But we're handling users who happily use FaceBook, WhatsApp, but all of a sudden complain that Zoom is not privacy preserving.

Paraphrasing it, but I think it's not too far from the truth of what most people outside of InfoSec think: "Not to mention an app where the government would be listening! I mean, Facebook is nice, they let us use their stuff for free. But the government is evil, as we have to pay them!"

Anyway - what is the DP3T stance on this, @gannimo ?

[oetiker]

@ineiti I totally get the preception problem ... but that is a marketing issue ... I guess disabling the mic function to start with would be a great thing :) and also only requesting access if the feature gets enabled ...

[ineiti]

I would love to try to implement such a feature ;) I'm looking for more than 2 years for a good excuse to do an audio communication between phones, but never got around to do it. There are some existing libraries, but those who seem to work are not Open Source. And doing some signal processing would be really great. Would make me younger by 20 years again ;)

[janbeutel]

hey tobi, long time no see/hear... what a coincidence. saw this on FB last night!

the austrian app (built by accenture, mandated by red cross, paid for by uniqua) started with only manual confirm buttons and then extended to BLE and audio chirps. that's a pretty simple and robust thing you can implement quickly. very pragmatic but requires like you noted proper positioning/marketing. problem is it's not possible in any way to protect this from "bad people/scenarios" as you lack channel capacity for proper codes, and there's a scaling issue. because of time of flight it's really good for ranging. but not for data transfer as well as multiuser separation you need some kind of coding (remember safe keys = long sequences, not short ones...)

also the audio ranging relies on two-way (sending it back and forth, distance is the time-of-flight/2 minus a constant? driver delay) and so far we are trying to do something that is purely one-way (for privacy/invasion reasons.

and then remember: the COVID contact tracing is NOT about distance estimation but about exposure between node pairs. (or larger constructs). so distance - yes its something we like to abstract this problem into. but really its about the exposure to aerosol and not about 2 meters or any other value with a length unit.

so yes there's a point and no it's not so straightforward.

personally, if it helps for COVID, i'd accept almost whatever it takes to solve it as long as the mechanism is transparent. but that last bit is opinion!

[oetiker]

@janbeutel the Austrian app ... (https://www.roteskreuz.at/site/faq-app-stopp-corona/) ... their FAQ entry about opensource ... when I read this I wonder ... if they should actually write ... the code is such a mess that we are all embarrassed to publish it ...but that's just my own 2c ... For the exposure being a complex topic, I totally agree, and the more information about the nature of the proximity we have the better ... maybe sound could even be used to measure distance if applied cleverly.

[ineiti]

@oetiker - see the above cited paper, where they manage to have around 10cm of accuracy using sound, even through pockets! As sound is so nicely slow, you can easily count how long it takes for a reply.

As far as I can understand, the paper does not make use of any advanced signal-processing theory, but merely uses threshold detection. One way to make this way better would be to use some preamble that can be detected much with much more detail than the beginning of a tone.

[ramsestom]

Already thought of ultrasounds to detect contacts or mesure distances in the case of COVID-19 but I think there is 2 major issues that have not been mentioned in previous comments: 1- Having devices that emit (near-)ultrasounds in every public places may cause health effects on animals and peoples (migraines, nausea, dizziness...) 2- The more devices there is at one place (so the more you are at risk of beeing infected by COVID-19), the more difficult it would be to separate the ultrasonic signal from one user to another. So the ultrasonic contacts detection may only work when a limited number of users are in range, which greatly limits its usefulness.

[ineiti]

@ramsestom - commenting: 1 - if you emit only once you receive a bluetooth-beacon, I think you can keep the emission frequency quite low 2 - remembering my work on wireless transmission, you can use spreading codes to transmit at the same frequency for multiple IDs, and still be able to separate them with great temporal precision

[ramsestom]

1 - if you emit only once you receive a bluetooth-beacon, I think you can keep the emission frequency quite low

It depends on what the ultrasonic signal is used for. If it is to detect contacts, then it can't be activated only when a bluetooth-beacon is emitted, it as to be emitted regularly. If it is to have a better estimate of the distance or the obstacles (a wall for example) between devices, then, you still have to use them regularly as users are not fixed objects (so the distance and obstacles between them change over time). Furthermore, encoding a signal with ulrasounds requiere some quite "long" signals (you can't encode as much data in a 1 second ultrasonic signal as in a bluetooth one). So the emission length and frequency would probably not be "quite low". Anyway, for animals like dogs, that are able to perceive these ultrasonic signals, it would probably disturb them. And for humans, even if they can't hear it, ultrasonic signals are still "processed" by the brain somehow and a prolonged exposure to ultrasounds have been proven to induce biological effects. So having hundred of devices that emit ultrasonic buz regularly everywhere you go might be a problem and could tire your brain with effects like nausea or dizziness (I don't say it would be the case for sure but I am not sure it would be a risk I am ready to take).

2 - remembering my work on wireless transmission, you can use spreading codes to transmit at the same frequency for multiple IDs, and still be able to separate them with great temporal precision

I am not an expert on this field but if you want to separate near-ultrasonic signals temporaly, that means that you will have to spread each signal over time. So the overall emission time for a signal will be even longer -> back to problem 1. Furthermore, due to the wavelength of the ultrasounds, it will still saturate quite quickly (from a certain number of signals at the same time it will be extremely difficult (or impossible) to separate them temporarily), much more quickly than with a bluetooth signal

[ineiti]

Back-of-the-hand calculation for the signal spreading - might be off a bit ;)

• 1kHz bandwidth (44kHz sampling rate, around 20kHz, before the cut-off filters)
• QPSK -> 2kbps
• spreading factor of 16 -> 16 orthogonal codes, can detect 16 different transmissions at the same time
• random choice of code -> 50% chance of a collision when 4 phones send simultaneously
• 1 second transmission time: 2kb / spreading factor of 16 -> 128 bits
• 128 can be enough to encode part of the EphID plus some timing information (local reception and transmission delay of the phone) - the full EphID must be known through BlueTooth

The protocol might be something like:

• phone A emits BT, starts listening on the mic
• phone B receives BT, transmits audio-ping with the received EphID for 1 sec, starts listening on mic
• phone A receives audio-ping and replies instantly, putting his delay-parameters in the 'pong'
• phone B receives the pong and can calculate the distance using the delay parameters
• phone A and B stop listening on mic

Of course no idea whether this would work or not ;) Also, currently Apple and Google are putting the BT-transmission in the system, so it might not be feasible to link the BT emission/reception with audio-transmission.

This would give 2 pings of 1 sec duration. If you can go beyond QPSK, you could lower the transmission time. Or transmit less information. So this would not be a constant background noise, but only sporadic ping-pong. I saw some systems that even put it in the audible spectrum with some nice noise - then you know what's happening...