marq24 / UUID0xFD6FTracer

Scan your current environment for devices running an app that make us of the ExposureNotification Service
GNU General Public License v3.0
122 stars 6 forks source link

Show RSSI of active beacons #12

Closed lmasellis closed 3 years ago

lmasellis commented 3 years ago

A list of active beacons with their associated RSSI signal strength would be useful for several purposes, e.g. to check how many beacons are actually in close proximity, close enough to be considered as dangerous contacts by the official apps, and how many are instead weak and not significant (e.g. behind a wall or across a street).

marq24 commented 3 years ago

The app does not try to give you any indication about a potential risk of an infection... That's the core functionality of the existing Corona Apps - or better the ExposureNotification Framework - they know so much better the possible additional side effects of a "good vs. bad" signal strength (e.g. phone in your back pocket vs. in your hand) and when to consider a contact as "close" and when this is not the case.

Showing the pure "current" signal strength for all beacons would IMHO have no additional value for the user - since I hope you can judge with your existing bio sensors much better if somebody is close to you or not. The only thing the app will tell you is IF the people around you are sending a signal that can be detected by your installed Corona App. No matter if he is currently "close by" or just on the other side of the street.

This is of course my personal opinion/experience - I have to admit, that since March 2020 I have not been at any place with a lot of people (simply cause I am in a very comfortable position - I don't have to) - so it could be that I totally misjudge the situation you are facing every day (e.g. in a city). So I am totally open minded to hear from you, what you exactly expect from seeing the signal strength of each beacon - or better I would like to know, what will you actually do based on this information - what's going to change in your behaviour based on the pure signal strength (that IMHO does not have to have anything in common with physical distance)?

lmasellis commented 3 years ago

No change in my behaviour based on the RSSI. By the way, the same is valid for the whole app: I installed it just to get more insight on the internals of the exposure notification system, which unfortunately it's not shown by the official national apps.

Seeing the total number of contacts provides you an idea of what the official app sees, but no information about how many of those contact would be actually considered by the exposure notification system. For example, the Italian app filters out beacons with RSSI<-73 dB; while standing in a room with a known number of active beacons, it would be interesting to understand if any more beacons (e.g. behind a wall) are considered "potentially dangerous" by the exposure notification system, even though they are not actually dangerous in the physical world.

To summarise: personally, the only purpose I find in this app is to get more insight on the inner working of the residue notification system, that is to get more information than what exposed by official apps. The more information you get, the better it is.

marq24 commented 3 years ago

... By the way, the same is valid for the whole app:

I truly believe it makes a change - at least in your head - but this starts to get philosophic here - let's skip that.

So let's get back to the facts/code - so when I understood you correctly - then it's not the actual RSSI you are interested - it's more that you would like to be able to define signal-ranges in which the beacons will be grouped (and count separately).

E.g. allow to create a group "smaller then -73db" : IGNORE, -73db < -50db WEAK, ... would that be an option?

how many of those contact would be actually considered by the exposure notification system

Since Apple & Google have decided to implement this as BLACK-BOX nobody can tell this for sure [very BAD decision IMHO - since it would allow more people to verify the results and get confidence in the overall functionality]. Instead we are all just guessing.

To summarise: personally, the only purpose I find in this app is to get more insight on the inner working of the residue notification system, that is to get more information than what exposed by official apps

If you just be interested in sniffing BT(LE) signals - then for sure apps like nRF Connect (from Nordic Semiconductor) would be way more useful -> https://github.com/NordicSemiconductor/Android-nRF-Connect

lmasellis commented 3 years ago

So let's get back to the facts/code - so when I understood you correctly - then it's not the actual RSSI you are interested - it's more that you would like to be able to define signal-ranges in which the beacons will be grouped (and count separately).

E.g. allow to create a group "smaller then -73db" : IGNORE, -73db < -50db WEAK, ... would that be an option?

Yes. That would be useful. But it would be also interesting to have a live listing of active beacons, ordered by signal strength and showing for each an "age" (time since last reception). That list would enable to get an idea of how many beacons are actually near, strong and stable, and how many are weak and fading or sporadic. It would also allow to have an idea of the effect of collisions in very crowded areas.

how many of those contact would be actually considered by the exposure notification system

Since Apple & Google have decided to implement this as BLACK-BOX nobody can tell this for sure [very BAD decision IMHO - since it would allow more people to verify the results and get confidence in the overall functionality]. Instead we are all just guessing.

Not really true: the evaluation of each contact is done inside the Google black box (really bad thing, I agree), but based on classification parameters which are determined by the national app and passed on to Google Play Services. Since national apps are mostly open source and/or get their configuration parameters from public APIs, we have ways to know how contacts are classified in each country based on signal strength and duration. See for example here: https://github.com/immuni-app/immuni-documentation/issues/112

marq24 commented 3 years ago

Yes. That would be useful. But it would be also interesting to have a live listing of active beacons, ordered by signal strength and showing for each an "age" (time since last reception). That list would enable to get an idea of how many beacons are actually near, strong and stable, and how many are weak and fading or sporadic. It would also allow to have an idea of the effect of collisions in very crowded areas.

Then I really would recommend to use nRF Connect or RaMBLE for exactly this purpose (record signal strength over time)... Nevertheless allow the user to specify at least a trashold in UUID-Scanner would be a good addon (for experienced users)

Not really true: the evaluation of each contact is done inside the Google black box (really bad thing, I agree), but based on classification parameters which are determined by the national app and passed on to Google Play Services. Since national apps are mostly open source and/or get their configuration parameters from public APIs, we have ways to know how contacts are classified in each country based on signal strength and duration.

I am aware the ExposureNotification Framwork can be configured individually by country - I just can guess, what kind of impact the provide params will actually have inside Google's BlackBox. But why it accepts different parameters anyhow? What effect/impact these parameters will have - and is there any kind of scientific based evidence we have what are "the correct"/"the best" parameters? Why we should use in Italy different parameters then in Germany? I would assume that weather/temp condition will have much more impact then nationality?

My impression is, that we are all just running a large beta test for Apple and Google in order to provide them with data that they will sell us again in the future. But at the end of the day I see way more benefit for us all by using this technology. It would be cool if countries all over the world join forces in order to develop a own independent and open Framework - instead of rely on companies (which obviously need to earn money).

lmasellis commented 3 years ago

Then I really would recommend to use nRF Connect or RaMBLE for exactly this purpose (record signal strength over time)...

True if you want to to serious formal research on the topic, and you are willing to carry specialised equipment with you in an underground trip, then do post-processing. If you just want to "get the feeling" in a casual way using just your smartphone, a live listing would be much more attractive. I am thinking of something like the list provided by the open source WiFiAnalyzer app on FDroid, yet much more simplified (id, rssi and age would be enough for each record). By the way: you are the author, so you decide. Thanks for the app you provided, if you feel my suggestion is useful and you want to implement it, it will be great; if you don't, well, that's ok anyway ;-)

Nevertheless allow the user to specify at least a trashold in UUID-Scanner would be a good addon (for experienced users)

Agreed.

My impression is, that we are all just running a large beta test for Apple and Google in order to provide them with data that they will sell us again in the future. But at the end of the day I see way more benefit for us all by using this technology. It would be cool if countries all over the world join forces in order to develop a own independent and open Framework - instead of rely on companies (which obviously need to earn money).

Totally agree. The EU totally thrashed the opportunity to go with a privacy-friendly solution when every country pursued its own app, which is total non-sense: no single country has enough power to really impose something to Google and Apple, so they got the chance to figure out their own closed solution and make sure it was the only one really viable for countries, so that they would be forced to accept it. Look at Switzerland: they developed a totally open solution (which was substantially copied by Google and Apple in their framework), but then they were forced to move to the G&A framework in order to be sure it would actually work on the phones...

marq24 commented 3 years ago

True if you want to to serious formal research on the topic, and you are willing to carry specialised equipment with you in an underground trip, then do post-processing. If you just want to "get the feeling" in a casual way using just your smartphone, a live listing would be much more attractive. I am thinking of something like the list provided by the open source WiFiAnalyzer app on FDroid,

I always thought, that apps like RaMBLE would exactly provide such a lightweight solution? - At least I can say, that nRF would show me a signal strength graph per found device on my phone... But I understood the message - you want something "easy to use" with a simple GUI... So I'll guess allow the user to deine "signal strength ranges" and group them (count separately) will be a acceptable approach... (I honestly don't want to list each found device separately from each other)

By the way: you are the author, so you decide. Thanks for the app you provided, if you feel my suggestion is useful and you want to implement it, it will be great; if you don't, well, that's ok anyway ;-)

Of course this is an option - another one is, to fork the repro ;-)

When I started with this app I wanted to keep it as simple as possible - in the meantime there are already multiple options & settings - I have the 'fear' that this all will end in the same state my GPSLogger II app is right now - every time when I could not decide myself for option A or B I have added a setting to give the user the choice... which ends into the situation, that users needs to be a hardcore experts in order to understand all the side effects (but all this might be just lame wining from my side) - writing all this might have even took longer, then actually just implement your suggestion ;-)

marq24 commented 3 years ago

hi @lmasellis I just have completed the initial impl of a RSSI grouping...

https://github.com/marq24/UUID0xFD6FTracer/blob/7cec98ac671c67d0644a2e235d8fc7994d3ae82c/app/src/main/java/com/emacberry/uuid0xfd6fscan/ScannerService.java#L361

Of course I have selected the initial PERFECT, GOOD, WEAK, BAD values wilde guessing - according to your input that the Italian Corona App use -73db as threshold... While I was testing the initial code I had to realize, to get a strength above -73db I have to put the sending phone (beacon) very close to the receiving device... Approximate in a range of less then 30-40cm... When I pt the sender and receiver next to each other the strength does not get higher then ~ -50db

So do you have suggestions for some initial "value ranges"... I have to admit that I am quite surprised, how close you have to be in order to get into the -73db range...

lmasellis commented 3 years ago

Seems like the value used to classify the beacons is corrected based on the following formula:

Attenuation = TX_power - (RSSI_measured + RSSI_correction)

where TX_power is reported in the encrypted payload received by the beacon (which derives it from a calibration table based on the device model) and RSSI_correction is the receiving end correction (also derived from the same table, but on the receiving device).

We cannot decrypt the payload, so TX_power cannot be found on a per-beacon basis. Also, it would probably be beyond the scope of this app to implement a table lookup for RSSI_correction. So one could think to just use averages over the whole table to get reasonable values, which would result in:

Attenuation = -22.93 - (RSSI_measured + 3.54 ) = = -26.47 - RSSI_measured

So if we set Attenuation = 73, a reasonable measured RSSI threshold would be somewhere around -100 dB.

Using two devices of the same brand at 1 m distance I get values around -75 dB, while at around 5 m I get values around -96 dB (btw, I used Ramble, nice suggestion!). So -100 dB seems to make sense.

Obviously if you get two devices which are wildly our of average, the result may be inaccurate, but I believe this would be not really relevant given the purpose of the app.

Reference: https://developers.google.com/android/exposure-notifications/ble-attenuation-overview

[Edit: corrected swapped averages and wrong signs in calculation]

marq24 commented 3 years ago

Good Morning!

when I saw the formula I instantly tried to get some additional information about that [without continuing to read the complete 'reply' (a very bad habit - but I paid the price for that!)] :-)

I came exactly across the same information - encrypted data in the payload etc... and how to find those values - and then I came across the en-calibration-2020-06-13 table... On the one hand side I thought about just using it (google use all my data all day long) - but then I also realized, that it's just like some static correction that then can be hardcoded anyhow... So after all that happend I came back here and read the rest of your reply - which gave me a big smile on my face. You might know these moments when yo realize that you are just made a moron out of yourself?

So what can I say? Thank you very much for all the details!

Just allow me to ask again, what's in your opinion a good approach?

lmasellis commented 3 years ago

I had a second thought on this, reading again the references above, I realized that -100 dB is practically at the bottom of the sensibility range for an Android phone, so the choice of -73 dB by Italian authorities without intermediate risk levels means they have given up the idea of estimating distance by signal power.

I am still convinced that it would be interesting counting at least strong and weak signals separately. A reference for this could be the Swiss configuration, which uses a three-bucket (plus a fourth for the bad contacts) classification:

Near: -infinity < attenuation ≤ 55 dB Medium: 55 dB < attenuation ≤ 63 dB Far: 63 dB < attenuation ≤ 73 dB Bad: 73 dB < attenuation

If we apply average corrections from the table, we get RSSI boundaries at -100 dB, -90 dB, -82 dB. But since for practical purposes -100 dB is the bottom of the scale, I would recommend either not providing a "bad" bucket (including it in the far one), or using something between -97 and -99 for this bucket.

marq24 commented 3 years ago

It took a while to integrate all this into the final app...

Screenshot_20200926-163903

Screenshot_20200926-163845

Are you able to compile the app yourself? [getting the branch] - if not I will upload a pre-release build here to github.

ljl-covid commented 3 years ago

This looks good! If it would be possible to tack in a last feature before pre-release, I think something would go hand in hand with this: could you should the number of BLE-enabled phones that are not broadcasting the beacon? Perhaps even as percentages: "40% are broadcasting, 60% aren't broadcasting". This would be useful to get an idea of how many people with smartphones around you have the apps, compared to those who don't.

marq24 commented 3 years ago

@lmasellis - just created a build - please check/update to https://github.com/marq24/UUID0xFD6FTracer/releases/tag/0.9.1.12

@ljl-covid - No clue what you have in mind, then you write "counting BLE-enabled phones" - even if I would had an idea how such a function could be implemented (counting something that is not broadcasting) - I would not do that.

ljl-covid commented 3 years ago

@marq24 thanks for the build. I mean any devices that are broadcasting any other BLE data; of course if they are simply not broadcasting anything, or their Bluetooth is off, they won't be counted, but I suspect many devices with BLE will be broadcasting something (as visible with BLExplorer or one of the many other apps); I just think it would be interesting to know which proportion of all devices that are broadcasting BLE stuff are broadcasting 0xFD6F in particular, because it would give a rough measure of the COVID app's adoption.

marq24 commented 3 years ago

@ljl-covid I don't see any way to count "mobile phones in range" - simply counting BTLE devices does not have anything in common with "potential covid app users" (aka "phones") - that's a task you have to do with your bio sensors.

Just to give you an example: I have here 1 x Phone, 1 x Smartwatch, 1 x iPad, 1 x Fitness Tracker, 2 x different Headsets and then from my road bike: 2 x PowerMeters & 1 Di2 Shifting... So that are in total 9 different BTLE Senders - while only 1 0xFD6F transmitter (and not taken into account that a phone can easily start to create plenty of BTLE-Signals)... So that would give you a 1:9 ratio - this would not tell you anything.

marq24 commented 3 years ago

@lmasellis can we clos this with the release of 0.9.1.13?

Citro12 commented 3 years ago

Hallo, prima Feature. Welchen Wert hat die deutsche Corona App als Grenzwert für "nah" in dBm ?

marq24 commented 3 years ago

@Citro12 nicht so einfach zu sagen...

https://github.com/corona-warn-app/cwa-app-android/blob/8913e9a92ce17d909d128422a67ccfd1ee9faf60/Server-Protocol-Buffer/src/main/proto/applicationConfiguration.proto#L62

Zu den dort gelisteten Werten muss man eben noch die "Korrektur" je DeviceType dazu rechnen... ich habe die jetzt mit dem Wert "22" hardcoded...

    RiskLevel gt_73_dbm = 1; // A > 73 dBm, lowest risk
    RiskLevel gt_63_le_73_dbm = 2; // 63 < A <= 73 dBm
    RiskLevel gt_51_le_63_dbm = 3; // 51 < A <= 63 dBm
    RiskLevel gt_33_le_51_dbm = 4; // 33 < A <= 51 dBm
    RiskLevel gt_27_le_33_dbm = 5; // 27 < A <= 33 dBm
    RiskLevel gt_15_le_27_dbm = 6; // 15 < A <= 27 dBm
    RiskLevel gt_10_le_15_dbm = 7; // 10 < A <= 15 dBm
    RiskLevel le_10_dbm = 8; // A <= 10 dBm, highest risk

Wenn ich meine beiden Android Telefon direkt aufeinander halte komme ich auf einen -dBm Wert von ~ -50... näher kann man sich da nicht kommen - also maximal komme ich auf eine RiskLevel von 4... die Werte 5,6,7 & 8 halte ich für nicht erreichbar - ABER ich habe mich da auch nach wie vor nicht Inhaltlich intensiv mit auseinander gesetzt

Citro12 commented 3 years ago

Dankeschön

Citro12 commented 3 years ago

Wenn ich meine beiden Android-Telefone direkt nach unten halte ich auf einen -dBm Wert von ~ -50 ... kann kann man sich da nicht kommen - auch maximal kommen ich auf eine RiskLevel von 4 ... die Werte 5,6,7 & 8 halten ich für nicht gehörtbar - ABER ich habe mich da auch nach wie vor nicht Inhaltlich intensiv mit teilhalten

Das kommt auf die Geräte an. Nähere ich mich einem iphone SE, geht der Wert bis -36 dBm. Bei einem Samsung S8 geht es gerade auf -75 dBm, bei 1 Meter Entfernung kaum unter -80 dBm. Die Geräte werden wohl unterschiedlich stark senden

lmasellis commented 3 years ago

@lmasellis can we clos this with the release of 0.9.1.13?

Sorry for the late reply. I tested the new version and it does the job. Despite a little bit of flickering (which is not this app's fault), you can get a clear idea of the number of active beacons in the immediate proximity. Nice work, thanks.