Physical web experiment not successful

norhanmhmd commented 8 years ago

Hello All we did a trial by planting a beacon in a fairly crowded closed restaurants zone the beacon was planned to send information about offers and menus of the restaurants we estimated that mobile users passing by the beacon daily to be 6000 and by calculating percentage of android users in the area and another percentage of people with bluetooth on , the estimated no. of people receiving the message were pretty good

we have been running the beacon for a week now and the result unfortunately is ZERO users opened the URL(although we checked the beacon is working fine and it only works for few mobiles that has physical web settings on)

I have been experimenting a lot with eddystone URL on different mobiles i have few challenges: Use Case:

when the objective is to notify the users that they are near something broadcasting information, physical web does not serve this objective, the user need to have the phone screen turned on and sometimes for up to 4 or 5 minutes in order to receive the message which is not practical in real world applications (in addition to having bluetooth on)
the other option is to put flyers or posters explaining the physical web and how to turn it on and wait for the notification which in most cases not practical,

My questions: is there something to be done to improve the implementation of the scenario above? is there a way to know how many people received at the least the prompt that explains the physical web ? do you have statistics on number of people with physical web settings on? Do you have plans to turn on physical web settings by default ? as i am wondering what is the point of having this option in android phones if the user is absolutely unaware of it

Thanks in advance Norhan

ferencbrachmann commented 8 years ago

Based on our experiences this does not seem right. Are you 100% sure that you there were no technical issues during the trial period?

norhanmhmd commented 8 years ago

Yes, when i come near the beacon with the mobiles i have been testing on (that have the phy web setting on) it works fine ( although as i said sometimes the screen need to be on for a couple of minutes before the URL is displayed), we also tested the bluetooth signal strength at different points within the beacon reach and it was good,we have set the advertising frequency as recommened for the physical web My guess that the problem with the use case itself that it requires the user to have the bluetooth on and to hang out within the beacon reach with their screen on for at least 4 to 5 minutes, and then agree to enable the physical web, which seems like a long shot

Do you have a similar use case where you had a better success ratio? can you please share with us the numbers?

thanks

scottjenson commented 8 years ago

There appear to be two separate issues here. The first is the 4-5 minute delay. That sounds like a bug. Scanning always takes place on screen on and should show w/in seconds. In fact, the scanning ONLY happens at screen wake time so leaving the screen on for 4-5 minutes should never result in it showing up. If, however, the screen went to sleep in the period of time and you had to turn the screen back on, that would make more sense. In effect, you were scanning multiple times and it eventually found the beacon.

Now, why it took multiple times I don't know. It certainly could be a bug in our scanner. But of course, we test this quite a bit and it's not something we're seeing here. When things do not show up, it's often poor data coverage. If you have a very poor/slow data rate, it can take many seconds to contact the PWS. If you don't have data coverage, that round trip won't happen (or happen too slowly) and the beacon will not show up. The other issue is the phone. We've noticed that a few phones (the oneplusone and the Nexus 5x) both have poor BLE stacks and can frequently miss scans. This is unlikely the cause of your problem, but it's the next likely thing I'd look at.

However, the second point I want to address is your expectation that people, without ever hearing about the Physical Web, will just walk into your area and pull out their phone to look for your beacon. We are not a push technology (because we don't want to create the worlds most annoying spam system) which means users have to want to take out their phones. This clearly means that users, who don't know about the Physical Web, are unlikely to do this. We see this much like wifi was in the early 90s. Coffee shops early on, needed big "Wifi inside!" signs so their customers knew they had to open their laptops and sign on. Today, it is just expected, and the signs have disappeared. We've long suggested that early PW deployments need to have a poster, sign, email, something to tell users to get out their phone.

Now I appreciate that a poster telling them to 'detailed instructions' is likely a bit much. This is why we've made the on-boarding (at least on Android) painless. You just need to look in the notifications tray and it will be there.

ferencbrachmann commented 8 years ago

Yes, even complete "blind" tests produce some measureable results. Maybe the mobile network coverage bad at your test site? Can you reveal the aproximate location? Maybe in sites with only 2G coverage (India comes to mind) experience challenges with providing rapid beacon notifications. Are you 100% positive that your analytics stack is working? Having 0 pageviews would lead me to suspect some issue in that domain.

louwng commented 8 years ago

Hi Norhanmhmd

Thanks for posting this as I am having similar frustrations with getting PW to work on Android phones.

Can i ask that we form a collective that will further test and share our experiences and ideas in order to resolve this issue, with the assistance and guidance from the Google Physical Web (PW) team off course.

I have already lost 3 potential customers as a result if this and one of them is a large Golf Shop franchise in South Africa with outlets country wide, they all say the experience is heavy friction and not seemless which is a pity as this would have been an ideal use case to implement.

Can I ask that Google PW give as the prerequisites similar to what they used to test PW and getting the results they got so that we can eliminate any setup or configuration issues on our side as a possible cause.

Love to hear your and Google PW comments

Regards Nico

scottjenson commented 8 years ago

@louwng we've already had this conversation on Twitter. There was apparently some confusion around Chrome vs Android OS support. If you're pushing people to use Chrome, that no longer works and it's not surprising you're having some problems.

It's now supported in Android OS and the primary requirement is having BT and location turned on. The moment you do turn it on, the notification should appear. If that isn't happening, something is wrong. Please tell us more and we'll try to help you. However, as others have commented, do make sure you have data as without that the Physical Web (or the web for that matter) won't work.

norhanmhmd commented 8 years ago

Hello All,

I tested on samsung S7 with bluetooth on & the physical web prompt did not appear (bluetooth signal strength was -75dbm),then we turned the physical web on & the url still did not appear the URL analytics are working fine, it displays the entries from the mobiles that i test on (is their another analytics platform for URL beacons?) Mobile network in the area is 4G, we also connected the above mobile to wifi still no msg appeared I am planning to go to the beacon location on thursday to test on more mobiles

louwng commented 8 years ago

@scottjenson apologies for possibly repeating myself above, I am trying to get an understanding of the different technologies in the BLE and PW space and how they fit into the Physical Web picture I was not aware of the changes of late with Chrome taking a back seat when Nearby is activated on your Android phone.

That is why I got confused between Chrome and Nearby, I was under the impression that Nearby was just an added feature and that Chrome was still the default way to detect Eddystone enabled beacons.

I actually only realised that earlier, what you said above, when I was reading thru the posts on this Github. That is why i was requesting Google PW team to provide us with default settings and configs, hope that makes sense and provides an understanding of the context in which I was asking this question.

Am I correct in saying that for Android phones before Android V6 (where Nearby is not available) one would still use Chrome as the default for scanning for Eddystone Beacons?

Please advise

scottjenson commented 8 years ago

Sorry, but I need to clarify a few of your points. What do you mean "turn the Physical Web on?" All you should have to do is have BT and Location on. Here is how I'd test this:

install nRF Connect and make sure you can see your beacon
put the URL in the beacon into verify.physical-web.org and make sure it passes
Open the Physical Web app and see if it shows up in the list view

If all 3 of these work, you should be in good shape, the notification should appear. However, just to be extra clear, location needs to be on and the scan only happens when you 'wake up' the phone.

scottjenson commented 8 years ago

@louwng the process is automatic. If you have the latest Google Play services, the Nearby notification will kick in and Chrome will no longer work. If Nearby is not available then Chrome will 'wake up' and start working. That is how we handle older phones.

Honestly, we are expecting Physical Web deployments to just use Nearby. We are trying to transition away from Chrome for notifications. To try and explain a mixed environment is, of course, very hard to do. Are you saying in your part of the world, that is is a 50/50 mix of phones with and without Nearby? If so, I would agree that isn't easy and no amount of 'settings guidelines' are going to help. However, as Google Play services goes very far back, I was under the impression the most Android phones will have it?

norhanmhmd commented 8 years ago

@scottjenson thanks for the steps above, i will go through them to make sure all is working properly i meant by turning the physical web on is change the physical web setting in chrome to be on instead off

Question: are most android phones need location on as well, because my mobile Huwaei P8 recieves the URL with only BT on?

thnx

louwng commented 8 years ago

@scottjenson thanks for the clarification on Nearby vs Chrome , much appreciated.

As far as the split between Nearby and Chrome is concerned, I must be honest, I don't know the exact percentage split but my guess would be more of a 70 / 30 or 60 / 40 split were Chrome is the higher percentage present at the moment.

Which Chrome version is the preferred version for BLE and PW where applicable of cause (ie: nearby not yet available on the user's phone)?

Most of the people I talk to on BLE and PW don't have Android V6.x yet on their phones.

I think that is what is making it difficult for us to get traction in the local market here, I have come across a few people that have looked at BLE but have aborted it due to the high friction of initially getting the user's android phone to pick up any PW activity at all.

However I personally believe that the technology has great potential and requires a lot of user education initially especially in my country as most, if not all, people that I have come across don't even have their bluetooth enabled on there phones.

Regards Nico

On Tue, Sep 6, 2016 at 4:52 PM, Scott Jenson notifications@github.com wrote:

@louwng https://github.com/louwng the process is automatic. If you have the latest Google Play services, the Nearby notification will kick in and Chrome will no longer work. If Nearby is not available then Chrome will 'wake up' and start working. That is how we handle older phones.

Honestly, we are expecting Physical Web deployments to just use Nearby. We are trying to transition away from Chrome for notifications. To try and explain a mixed environment is, of course, very hard to do. Are you saying in your part of the world, that is is a 50/50 mix of phones with and without Nearby? If so, I would agree that isn't easy and no amount of 'settings guidelines' are going to help. However, as Google Play services goes very far back, I was under the impression the most Android phones will have it?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/physical-web/issues/816#issuecomment-244975874, or mute the thread https://github.com/notifications/unsubscribe-auth/ATTb5XQjlgyW_YSTIZmwVNKP0mtkUq-Zks5qnX5DgaJpZM4J0VW9 .

ferencbrachmann commented 8 years ago

@louwng you don't need Android 6, you only need Android 4.4+ There should be no "split" you mention: Nearby Notifications is in Google Play Service and that's usually bundled with Chrome (and the rest of the Google Apps) in all big global mobile markets exept China.

louwng commented 8 years ago

Hi Ferenc

Thanks for clarifying this.

Is there somewhere were i can read up more on Android vs Chrome vs Nearby and how they all fit together or even an infographic that summarises everything and so the relationships between them all?

Regards Nico

On Wed, Sep 7, 2016 at 12:39 PM, Ferenc Brachmann notifications@github.com wrote:

@louwng https://github.com/louwng you don't need Android 6, you only need Android 4.4+ There should be no "split" you mention: Nearby Notifications is in Google Play Service and that's usually bundled with Chrome (and the rest of the Google Apps) in all big global mobile markets exept China.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/physical-web/issues/816#issuecomment-245242055, or mute the thread https://github.com/notifications/unsubscribe-auth/ATTb5fhRuNKw8v1A88SDgLn5Z-s5iycTks5qnpRLgaJpZM4J0VW9 .

scottjenson commented 8 years ago

@louwng we'll be updating our docs at physical-web.org soon. But it's not really that complicated:

Notifications were in Chrome
They have moved to Android

In either case, you need to have BT and Location on

rlaudebat commented 7 years ago

Hi Even though I undertand that you want to avoid having PW being a spam box, during the first discovery of a beacon the metadata of the website is not show on the notification. Moreover even if they click on the notification they are redirected to the proximity menu where a long speech is displayed and the link of the website sits in a small box down the window. From a custumer experience perspective I think it is not trivial to understand what's going on Would it be possible from your point of you to either display the metadata of the website even for the first discovery or rework the proximity parameter window to make it more understandable? Thanks Remi

cgorrell commented 7 years ago

I'd like to just add that I ran a similar experiment recently with a large restaurant venue. From tracking the URL, we got an estimated 850 notifications pushed out (not a super accurate number when you take into account cacheing) with only 4 visits to the website. This was over the course of a week with upwards of 20,000 visitors. During the week I tried several different title, image, and header combos to see if the results would improve. Those changes did not appear to have an effect.

My conclusion was that the onboarding process for getting someone to opt-in just by seeing the device nearby in the notifications panel has too much resistance.

As far as use cases go, I don't see an immediate opportunity to use physical web as an informative / promotional tool because of the onboarding resistance to nearby (even if I did put up big signs in the venues asking people to opt-in).

In terms of using physical web as a tool for creating interactive experiences where the ability to engage with a product or service is significantly limited without physical web / beacons, I can see people opting in then with the assistance of education by digital signage and such.

To be clear, I see huge potential down the road for using these devices as a promotional tool, but it is going to require less friction to opt-in to the physical web, or we'll just have to wait for enough people to opt-in from an interactive use case that promotional use can take advantage of the new population using the service.

Just my 2 cents after testing eddystone-urls on some beacons in the field this week.

scottjenson commented 7 years ago

I've often compared the Physical Web to the early days of wifi. Initially, no one knew what it was or why it was needed. Users were initially told to type in an "SSID", large "wifi inside!" signs were needed to let people know, and in general, it took a while for the overall understanding how it worked to sneak in.

The Physical Web, initially, will require a strong, concerned effort to inform people. This is why we feel conferences are a great starting case. In my talks with a city in the UK that wants to roll this out, they clearly understood the need to come in with 3-4 experiences all in the same area to show the system was in place and it would encourage others to jump in.

I'm sorry your deployment didn't work but I would argue that as you are on the leading edge of early deployers, you'll need to push your initial users a bit harder to get them on board. As others join in, this will get steadily easier.

viviancromwell commented 7 years ago

hi I would love to understand a bit more how you use it and when. I found in addition to the new concept of opt-in notification, beacon itself would have to be presented at the right time with the right motivation. For us, we found steady usage of the app (ios) but not through the usage of beacon.

We are not implementing Physical Web today but interested in the future if it makes sense. so would love to chat more. Send me an email at viviancromwell@gmail.com.

thanks

al: www.angel.co/viviancromwell tw. www.twitter.com/viviancromwell li. www.linkedin.com/in/viviancromwell

On Sat, Sep 24, 2016 at 11:55 AM, cgorrell notifications@github.com wrote:

I'd like to just add that I ran a similar experiment recently with a large restaurant venue. From tracking the URL, we got an estimated 850 notifications pushed out with only 4 visits to the website. This was over the course of a week with upwards of 20,000 visitors. During the week I tried several different title, image, and header combos to see if the results would improve. Those changes did not appear to have an effect.

My conclusion was that the onboarding process for getting someone to opt-in just by seeing the device nearby in the notifications panel has too much resistance.

As far as use cases go, I don't see an immediate opportunity to use physical web as an informative / promotional tool because of the onboarding resistance to nearby (even if I did put up big signs in the venues asking people to opt-in).

In terms of using physical web as a tool for creating interactive experiences where the ability to engage with a product or service is significantly limited without physical web / beacons, I can see people opting in then with the assistance of education by digital signage and such.

To be clear, I see huge potential down the road for using these devices as a promotional tool, but it is going to require less friction to opt-in to the physical web, or we'll just have to wait for enough people to opt-in from an interactive use case that promotional use can take advantage of the new population using the service.

Just my 2 cents after testing eddystone-urls on some beacons in the field this week.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/google/physical-web/issues/816#issuecomment-249381329, or mute the thread https://github.com/notifications/unsubscribe-auth/AAPIN2z-kZHX4SgKy2cHjxI8JTa8J9brks5qtXIrgaJpZM4J0VW9 .

norhanmhmd commented 7 years ago

@cgorrell i would love to know how do u track the URL to know the number of notifications requested(u mentioned in ur example it was 850) , because what i am able to measure right now is only how many hits the url gets but the stage before that i have no data as for my use case we double checked the beacon is working fine, in 1 month the url got 17 unique page views which is way below our estimation the conclusion i reached that having the bluetooth and location turned on in addition to the screen on while being near the beacon are too many conditions in real life situations which decreases dramatically the hit ratio to make sure that my conclusion is accurate i need as well to measure how many people got the initial PW notification but did not opt in , till now i did not know we can measure that :)

Apprecite ur help

Thanks

ferencbrachmann commented 7 years ago

When was this experiment done? Is Nearby (Google Apps) installed on most of the phones in your target market? These numbers seem low even compared to our own modest data.

louwng commented 7 years ago

Hi @cgorrell

As @norhanmhmd and @viviancromwell has indicated I also would like to know how you managed to track the URL to know the number of notifications requested(u mentioned in ur example it was 850)

Please advise

Regards Nico

BeaconWorx - http://beaconworx.com Twitter: @beaconworx

On Monday, 26 September 2016, norhanmhmd notifications@github.com wrote:

@cgorrell https://github.com/cgorrell i would love to know how do u track the URL to know the number of notifications requested(u mentioned in ur example it was 850) , because what i am able to measure right now is only how many hits the url gets but the stage before that i have no data as for my use case we double checked the beacon is working fine, in 1 month the url got 17 unique page views which is way below our estimation the conclusion i reached that having the bluetooth and location turned on in addition to the screen on while being near the beacon are too many conditions in real life situations which decreases dramatically the hit ratio to make sure that my conclusion is accurate i need as well to measure how many people got the initial PW notification but did not opt in , till now i did not know we can measure that :)

Apprecite ur help

Thanks

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/physical-web/issues/816#issuecomment-249508022, or mute the thread https://github.com/notifications/unsubscribe-auth/ATTb5Qsr9pbbQ94dVkPKQlCtoPuWRXjIks5qt4CzgaJpZM4J0VW9 .

google / physical-web

Physical web experiment not successful #816