search

google / physical-web

The Physical Web: walk up and use anything
http://physical-web.org
Apache License 2.0
6k stars 665 forks source link

Android app doesn't detect BLE beacons if flags are included, but iOS app does (which is correct) #870

Closed uday-agarwal closed 7 years ago

uday-agarwal commented 7 years ago

Background: Eddystone protocol consists of two parts - "Complete List of Service UUIDs", and "Service Data". Now, the "Flags" field is not mentioned in this protocol, which could mean it may or may not be a part of the ADV packet (the example frame mentions it though). As per Bluetooth Core Supplement Spec (CSS) v5, Flags were mandatory, even for beacons, but starting CSS v6, Flags have been made optional for beacons (specifically, non-connectable ADV). Due to this, some devices still adhere to CSSv5 and thus mandatorily transmit flags as part of the packet.

Problem: When devices transmit flags as part of the Advertisement packet, the Physical Web app on Android does not show such beacons at all. However, the corresponding iOS app shows such beacons (which is the right way to do it). Third party apps such as Beacon Toy on Android also correctly show such beacons (when flags are present).

So the Physical Web android app needs an update to detect such beacons when flags are present.

scottjenson commented 7 years ago

Two different points: 1) the Physical Web app chokes if flags are set. This is a bug and should be fixed. Thank you! We'll get working on this.

2) The use of flags The flags in the Eddystone spec are mentioned, they just aren't used yet. However, any flags set must e part of the Eddystone spec and it appears you are setting your own proprietary flags. We do have a few ideas of what we'd like to use flags for but haven't really pushed them yet as the ecosystem is still forming. However, you proceeding on your own will likely invalidate your beacons once they are added to the spec.

Can you let us know what you're trying to do with these flags, they may line up with what we're considering for the space and hurry that proces along.

uday-agarwal commented 7 years ago

I am really sorry for replying late. Somehow I didn't see any notification when you replied, and forgot about this. :(

To answer your question, the silicon vendors enforce the presence of flags just because of the CSS v5 spec (as I mentioned in the problem statement), and so there is nothing really worthwhile happening with those flags. In other words, I am not setting any proprietary flags on my own. For example, I am using Nordic nRF51822 (with SDKv11 and SoftDevice s130) and I cannot disable the flags, as far as I know.

The value of the flags could be one of the following: 0x06 - "General Discoverable mode" bitwise ORed with "BR/EDR not supported" - true for most devices 0x0A - "General Discoverable mode" bitwise ORed with "Simultaneous LE and BR/EDR capable controller" - some devices support this

If we start supporting these flags in Physical web, the problem is solved. Looking forward to any further questions.

scottjenson commented 7 years ago

OK, two different issues here

Standard BLE flags We're doing nothing fancy and the standard flags should just work. If you're using a standard BLE flag and somehow we're not finding it, that's a problem.

Eddystone flags We have our own flags embedded w/in the Eddystone format. I confused your question and assumed you were using those. The bug I referred to was in this code and it has been found and fixed. However, I'm assuming this has nothing to do with your original issue.

So I guess we're back to what standard BLE flags you're using that is somehow breaking the Physical Web scanner? It would be helpful if you could run a test with X flag on and off, showing it working with our scanner and then not working with it.

uday-agarwal commented 7 years ago

If you're using a standard BLE flag and somehow we're not finding it, that's a problem.

That is exactly what I am talking about.

So there are two test cases:

  1. I used Nordic nRF51822 based Eddystone URL beacon, which has mandatorily enabled BLE flags at the beginning of the packet (byte stream - 02 01 06; same as the Eddystone protocol example). The sniffer screenshot shows the exact packet. This is detected by Beacon Toy app on Android as a valid Eddystone URL packet, but the Physical Web app shows nothing.

  2. I used the Beacon Toy app itself to advertise a Eddystone URL, which is detected by sniffer and also by the Physical Web app. Notice that there are no BLE flags (02 01 06) here at this packet's beginning.

These BLE flags at the beginning of an advertisement packet are dependent on the vendor, and the user can't do anything about it. For example, any android phone generated advertisement packet won't have these flags (I have tested this separately), but many embedded devices (such as Nordic) have these on.

Case 1 is the bad case (the physical web app should be able to see through the flags). Case 2 is the good case. If I use an iPhone, the physical web works fine there (case 2).

scottjenson commented 7 years ago

Sorry, but it would be so much easier to communicate/debug if you could just tell me the official BLE flag you're using that's causing the problem. That way I can reproduce the problem by turning it on here and testing.

adriancretu commented 7 years ago

The first URL redirects to a non-https website, that's why it's not showing up. On another note, it's weird that BLE flags are missing from the Android advertisement. What device are you using? If flags are missing and it still works, I guess we could have 3 more bytes for the URL? @scottjenson ?

uday-agarwal commented 7 years ago

@scottjenson I am using the official BLE flag value of 0x06. The first three bytes in my payload are - 02 (length of the Advertisement field) 01 (advertisement field type - Flags are indicated by 01) 06 (the value of the flags)

@adriancretu I used Moto G (can't tell you right now which generation; will have to check). In general I realized that when we use Android phones to advertise beacons (non-connectable advertisement), there is no presence of BLE flags. I used Android 6.0. That makes sense from Bluetooth Core Supplement Spec v6 perspective.

The first URL redirects to a non-https website, that's why it's not showing up.

Is that a defect?

If flags are missing and it still works, I guess we could have 3 more bytes for the URL?

I agree, although I believe we should have the flags supported for backward compatibility.

scottjenson commented 7 years ago

@adriancretu brings up a good way to test this:

  1. Beacon using 'standard' flags with https URL -> this should show up
  2. EXACT same beacon w/ new flags

We need to make sure there isn't something unrelated to the flags that are the issue: e.g. you have a 'bad' URL. This is why we always recommend verify.physical-web.org be used before you test any beacon.

uday-agarwal commented 7 years ago

I tried sending an https URL, and it works! Thank you. Is this a fundamental requirement for the physical web to detect the URL?

Side note: I tried this on 4 different phones (Xiaomi MI 4, Lenovo K4 Note, Google Nexus 5, Moto G 3), and the Physical Web app on all but the MI4 detect the beacon. On the MI4, Chrome is able to detect the beacon (via Settings->Privacy->Physical Web) but the Physical Web app itself doesn't detect it. Not sure why this would happen. Maybe some settings are wrong? Maybe there is a defect? I don't know. I have given all the permissions to the app though.

scottjenson commented 7 years ago

Eddystone beacons can broadcast any URL. The Google scanner only shows HTTPS as the Chrome team is very security concious and wants to prevent man in the middle attacks. This is why we always suggest you user verify.physical-web.org for any URL that you put into a beacon.

As to the MI4 there a range of things to try:

uday-agarwal commented 7 years ago

For the MI4, I tried all the above. I am broadcasting every 100 ms. Android version is 6.0.1 (MMB29M). Other apps detect the beacon. Even chrome app shows it in the physical web section! It's just the physical web app which doesn't detect the beacon. The physical web app showed this beacon once to me (when I tried it the first time after changing to HTTPS; that's when I got really excited), but then it never showed up again.

uday-agarwal commented 7 years ago

OK I just killed the app, cleared the app data from settings, and started it again. And it works now! I guess we can close this now, unless you want to probe further. And thanks for all the help.