quaggaJS not working on iOS 12

[michaeltrollan]

The release of iOS12 appears to have stopped support for accessing the camera through Safari websites.

The camera does not load.

You can see the Quagga example not loading on iOS 12: https://serratus.github.io/quaggaJS/examples/live_w_locator.html

You can also see it not working for WebRTC samples: https://webrtc.github.io/samples/src/content/getusermedia/gum/

https://webrtc.github.io/samples/src/content/devices/input-output/

[michaeltrollan]

One possible clue is this article: https://medium.com/@tuure.kaunisto/ios-12-update-caused-the-webrtc-stream-in-our-home-screen-web-app-to-stop-working-7ab7916eef45

After the iOS 12 update the stream stopped working. ... There were no JavaScript errors and it just seemed that the fullscreen version dismissed the onaddstream event handler altogether. . A deprecation warning instead of silent failure would have saved us a lot of trouble but we did finally notice that the RTCPeerConnection.onaddstream property was deprecated along with the callback parameters for other methods. We fixed the issue by switching to the ontrack event handler and refactored the old callback code to use promises.

[ericblade]

Anyone here equipped to check this out? Without access to a live iOS device, i'm kinda useless for all the iOS issues around here.

[haycoin]

Anyone here equipped to check this out? Without access to a live iOS device, i'm kinda useless for all the iOS issues around here.

Hi Eric, I also need to have your marvelous app working on IOS. How can I help you, which test do you need on iOS 12 ? If i can help ...

[ericblade]

Actually, I might be able to look into some things, if it's possible to recreate the problem on a Mac. I have an old-ish Mac Mini, I'm not sure if Safari gets updates independently from the OS or not, though. If it does, then I could potentially use a current version of Safari on the Mac to check it out.

@haycoin it looks like the whole sample app is failing.. i'm also hearing that my web app is failing on iOS 12 (no idea if my webapp worked on previous iOS, as I had no users with it until a few days ago) .. getting some information about what is going on with it would be helpful.

[haycoin]

@ericblade Thanks for you quick answer. I tryed on last iphone (ios 12.1) with Safari & Chrome, the browser just don't ask to use camara permission, as it does on android. @michaeltrollan suggested problem linked to "ontrack" , did you try this ? If you tell me the way I can test, or output console etc ... i'll would happy to help.

[ericblade]

i haven't tried anything -- i don't have an iOS device. getting the output from the dev console would be pretty helpful.

Even Android's camera permissions are all sorts of messed up -- you don't get an authorization prompt to use the camera until after you try to use it. So, you can't enumerate the camera devices first, without having the user first try to use a camera.

I don't know if we're using anything that @michaeltrollan 's comment talks about or not.

[michaeltrollan]

Here's the output from the dev console:

NotAllowedError: The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.

Michael

[michaeltrollan]

I tracked down the issue to the exact call. Inside of /src/common/mediaDevices.js, there is a call to navigator.mediaDevices.getUserMedia(constraints). That call is returning an error in iOS 12 Safari.

Here I am including a screenshot of both iOS 12 Safari (which doesn't work) and MacOS Chrome 70 (which does work).

iOS 12 Safari:

MacOS Chrome 70:

Here is the snippet. Try it in the console of any browser you'd like to test.

var constraints = { audio: true, video: { width: 1280, height: 720 } }; 

navigator.mediaDevices.getUserMedia(constraints)
  .then(function(mediaStream) {
    console.log('getUserMedia completed successfully.');
  })
  .catch(function(error) {
    console.log(error.name + ": " + error.message);
  });

[ericblade]

so... how does one get the iOS browser to prompt for permissions? It would seem that it's not as "simple" as in Android, where you just try to use it, then it prompts.

I went to check and see if my old Mac Mini can install the latest Safari, and that looks like a big no -- Safari 12.x only goes to the two most recent versions of MacOS, which my old Mac isn't getting.

[michaeltrollan]

The iOS browser asks for permissions as soon as you try to access the camera. e.g. It would happen right on page load for this URL (if it were working): https://serratus.github.io/quaggaJS/examples/live_w_locator.html

I meant to warn you that MacOS Safari is not the same as iOS Safari, so even if your Mac Mini were working it wouldn't be a good test. It seems to be something related to the iOS Safari browser.

Hopefully you can get ahold of an iPhone. I'd love to have your help figuring what's going on!

[elitree]

FWIW, I am able to load the example page successfully in Safari on my iPhone 6s running iOS 12.1...

I don't see any JS errors in the console when running the sample page.

[ericblade]

@michaeltrollan @elitree

.... so, we've got one person with iOS 12.1 that is able to use it, and one that is not?

Correct me if I'm wrong here, but isn't getUserMedia() the correct way to get access to the camera, which should prompt for access?

What I tried to do with the application I'm working on, is enumerateDevices prior to Quagga.init(), and I've discovered that that only works on desktop -- on mobile, enumerateDevices does not trigger a camera permissions request, and simply returns an empty array. Therefore, it's a bit of a problem, in that you want to be able to select a proper device id, but you can't, because you can't know what the deviceIds are until after the user has granted permissions, which it can't do unless you just try to blindly open a camera without an id. So, from a user perspective, that kind of sucks.

That doesn't seem to be at all similar to what's going on here, though. Eli is seeing a proper request for camera permissions, Michael is not.

Unfortunately, my iPad was stolen out of my car when I was visiting silicon valley last year, and it's difficult for me to justify a purchase right now. (it's kinda funny if you consider that Detroit is supposed to be the #1 place to get your car broken into, but i've lived here for 20 years and never had a problem.. go to the richest place in the country for a week, and got hit twice.)

[a-kildahl]

@ericblade, I do not have a MacBook so I can't get the console output, but I can't start it neither on an iPad with iOS 12.0.1 in Safari nor Chrome. Works on Android and Windows.

[ericblade]

@elitree @michaeltrollan @a-kildahl could we all clarify exactly which url we are using that works? i feel like we're looking at different demos. of course, i'm still blind leading here, but i'm trying :-)

[elitree]

Hi Eric, I'm using https://serratus.github.io/quaggaJS/examples/live_w_locator.html .

[ericblade]

Thanks! That's the same URL that @michaeltrollan said isn't working for him. SO, I don't know how to go about finding out what the difference is here right at this moment. Ideas?

[schreck4761]

That url worked for me on iPhone XS iOS 12.1. For anyone it isn't working for, is there any chance you blocked the permission at some point? It won't ask for permission again after declining.

[schreck4761]

using safari

[mohshraim]

iOS safari work iOS chrome will fail as they use WKWebView and userMedia not supported on WKWebView Mac Safari and Chrome work

[ericblade]

That url worked for me on iPhone XS iOS 12.1. For anyone it isn't working for, is there any chance you blocked the permission at some point? It won't ask for permission again after declining.

how does one reset permissions if one has denied?

[michaeltrollan]

In my case, the url was working on iPhone X iOS 11 Safari. I upgraded to iOS 12 overnight, loaded up the url right after, and it wasn't working. I don't remember declining camera access (certainly not across all of those different urls that I listed), but it's possible and would be worth testing out the option of resetting permissions to see if that's the case. Like @ericblade though, I don't know how to go about doing that. Any ideas?

[schreck4761]

On further testing it does prompt me every time for permissions even if I hit cancel on iOS 12.1 on iPhone. In the Mac version of safari it doesn’t reprompt.

General iOS access to mic and camera can be found in Settings -> Safari -> Camera & Microphone Access. Maybe toggling that setting would help reset something messed up by updating to 12.

[michaeltrollan]

Good news: that setting appears to be the issue! I discovered that on both of my test iPhones, the setting had been switched off. Interestingly, prior to upgrading to iOS 12, the setting was not an issue -- the pages were loading. I certainly didn't change the setting, but now it's suddenly turned off.

My QA team reported the same issue with their test iPhones as well (that the pages weren't working). I'll have them look at the setting.

I hope that this isn't a common issue for the general public -- if the setting has been turned off by Apple, most users will not think to look at settings, and that will mean that most Quagga implementations in websites will not be working.

For now, I'm going to consider adding help text, something along the lines of "Please ensure that this setting is enabled: Settings -> Safari -> Camera & Microphone Access".

[ericblade]

A problem that I've discovered on iOS, is that it seems to be much less forgiving about constraints than Android and Windows Chrome -- my default configuration that works on literally everything I've tried it on, fails under iOS with "Error: Invalid Constraint". I was able to discover this by trying to write the error into a div using React, but React ended up giving me a super solid error dump screen instead, that is very helpful.

So, make sure your constraints aren't invalid, according to whatever Apple seems to think is valid.

[ericblade]

.... so, Android and Chrome seems to work just fine with width: 800, height: 600, for iOS, though, it's looking for a width: { min: 640 }, height: { min: 480 } to properly work.

[a-kildahl]

Quick update from me: After updating both an iPad and an iPhone to iOS 12.1.1 the issue were resolved. Maybe it were a bug in the OS, and not Quagga?

[mohshraim]

@a-kildahl working with you on Safari or Chrome?

[a-kildahl]

@mohshraim Safari.

[mohshraim]

@a-kildahl for me it is working on Safari before update

[nitinsbuzz]

It seems ios 12.01 had that problem, but it turned out to be a bug. Fixed in the new update. Mobile ios safari.

[michaeltrollan]

It seems ios 12.01 had that problem, but it turned out to be a bug. Fixed in the new update. Mobile ios safari.

Ah ha, that could certainly explain it. Do you have any articles about that, or did you find out by trial and error?

I'll give it a test in the latest iOS 12 and if it's working will close this ticket.

[nitinsbuzz]

I use quagga on a daily basis and i noticed the same thing when my ios was updated to 12.0.1. After that, did some trial and error, update, and it worked.

[SuperCreate73]

Not sure the issues are related but I made a basic page with live recognition. It worked fine just about everywhere, including an iPad with iOS 11 but just would not work on my iPhone 5S i just updated to iOS 12.1.3. As I started playing around with quaggajs just after updating I have no clue if it would have worked on iOS 11 on the phone.

[michaeltrollan]

@SuperCreate73 - Are you getting this as your error? NotAllowedError: The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.

If so, it may be related. If not, it's likely something else.

[SuperCreate73]

Yep, that's it. And after a lot of searching, trial and error I'm quite happy to say I found a workaround if this can help someone else or maybe for an update of the library. I think the

[michaeltrollan]

Great! Please do share the workaround here; I'm curious what you found out.

[SuperCreate73]

Sorry: I keep pressing enter to start a new paragraph before I finished my post..and it sends the post rather than let me make a paragraph. So the workaround has been added to my previous message...

[SuperCreate73]

Hope it's understandable as english isn't my first language.. :)

[michaeltrollan]

The english looks great. Can you add an example of the before and after HTML? It sounds simple, but it'll be useful for folks to see.

[SuperCreate73]

Sure..! The way I came to this is noticing the

Previously had, inside body: <div id="vid"></div> as a placeholder for Quagga and added this tag inside: <div id="vid"><video playsinline autoplay></video></div>

Glad if this helps!

[thbar]

FWIW, I also got the NotAllowedError: The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permissionerror message, and initially tried out other workarounds, after figuring out it was just because the site wasn't SSL enabled (https://stackoverflow.com/a/53486992/20302). In case in helps other people here!