jsQR freezes on iOS 14 Beta (Safari & Chrome)

[robertpriewe]

Hi, on iOS 14 Beta the camera shows up for a second and then freezes. Tested using iOS 14 Dev Beta 2 on iPhone X on both safari & chrome. Was working fine before on latest iOS 13

[digitalmind]

The Issue still persists on iOS 14 release version

[mauriziomannini]

Same problem here: I confirm the issue on iOS 14 release version. Tested on iPad Air 2 and on iPad Pro 2020.

[dansteingart]

Ditto. Same error on iPhone 11 pro in ios14. No errors thrown in the console.

[harry503]

Please someone take a look at it. It's broken :( on Safari IOS 14

[dansteingart]

I dug around a little bit on this. It feels like it has something to do with the video engine.

For example: the demo code uses this framework

    function tick() {
      loadingMessage.innerText = "⌛ Loading video..."
      console.log("Video Ready State = " + video.readyState)

      if (video.readyState === video.HAVE_ENOUGH_DATA) {
               // do stuff, check for QR, act on it as you wish, etc      
           }
  }

where tick gets called on each frame. I added the bit where it logs the video.readyState to the console.

video.readyState documentation here

The readyState is reflecting what we see. First it's 0 (video.HAVE_NOTHING), then 4 (video.HAVE_ENOUGH_DATA), then back to 0 ('video.HAVE_NOTHING`).

So it seems like an underlying video challenge. Will dig more.

[dansteingart]

Brief update: in the URL bar if you press the camera button to disable the camera, and then turn it back on, the video unfreezes and the QR code can be scanned BUT you have to do it before the little camera goes away. Really tricky.

[cozmo]

Ugh I have a love hate relationship with the demo page. It's nice because it serves as a very visual/hands on demo of the functionality of jsQR, and also it shows how simple the code to call jsQR can be, but the flip side is the getUserMedia APIs are a rabbit-hole of complexity, and making a solid use of them takes a lot of code to handle edge cases like this. In the meantime like 40% of issues opened against jsQR are actually webcam/demo page related issues that are not at all related to the functionality of the underlying library.

I'm not really sure what the best course of action is here, since personally my interest in building a robust getUserMedia demo is very low, but my interest in building a rock solid QR code scanner is quite high, and unfortunately those 2 things seem hard to separate.

That said, it seems like this iOS thing is coming up a fair amount, so probably worth seeing if there's a simple fix for. Some investigation was done on https://github.com/cozmo/jsQR/issues/157#issuecomment-683720278 as well, I'm curious if that could be helpful?

[dansteingart]

Thanks @cozmo . I just made a page that tested getUserMedia independently of jsQR and confirmed it has nothing to do with jsQR. #157 is a good lead, but the problem is that it doesn't throw an error as far as I can tell when it freezes.

[dansteingart]

Towards a work around. Through a console if I inject

    navigator.mediaDevices.getUserMedia({video:{facingMode: "environment"} }).then(function(stream) {
      video.srcObject = stream;
    }

The camera momentarily restarts, but then freezes again.

[amill123]

Consider dropping the video FPS. Decreasing frameRate in the video attribute worked for me. I was previously experiencing the same issues. Ideal frameRate of 10, whilst it increased the time before the freeze happened, still froze before I could get the QR code in front of the scanner.

navigator.mediaDevices.getUserMedia({ video: { facingMode: "environment", frameRate: {ideal: 5, max: 10} } })

[dansteingart]

@amill123 thanks, that did the trick. I dropped the ideal to 10/max to 15 as well and that also froze. I didn't think to further down, good job 👍 . @cozmo I know this is way outside of the library, but any sense what could be making the browser choke at these higher rates. This is all new to me.

[cozmo]

I don't, but I imagine the video readyState dropping to HAVE_NOTHING is related - perhaps the camera is struggling to push to requested FPS, and so the stream hangs? Unclear.

That said, I'd probably head down the route of diffing some of the https://webrtc.github.io/samples/ samples and the jsQR demo trying to figure out what causes the difference (assuming the bug wasn't present in the webrtc samples).

[soundreactor]

i managed to make it work comparing to the webrtc sample, 2 things i had to change.

1. the video element must have "autoplay playsinline"
2. the video element must NOT be set to visibilty:hidden. you can have it behind something to hide it.

[nettermensch]

Find attached some html-demo which is simpler than the demo, because it does not render the actual image through a Canvas, but directly through the video-element. And: the scanning is done via timeout... (I tried to reduce the code as much as possible). - Of course: the nice drawing of the red lines around the QR-code is missing now, but the detection of the code does work... Example works with iPad 14.

It must have to do with the issue that @soundreactor just reported: the video-element must be visible.

jsqr_test.txt

[joshbontrager]

After spending hours trying to track down what was causing this issue, I found a solution similar to what @soundreactor and @nettermensch recommended!

My problem stemmed from creating the element in JS without ever inserting it into the DOM. I finally just added the following video element to my page. Querying the element to set the srcObject from getUserMedia() magically fixed everything! <video autoplay muted playsinline style="position:absolute;opacity:0;top:0;z-index:-1000"></video>

I think Safari messes with the video ready state if it doesn't think the user can see the video element, and I was drawing everything out to my canvas. I think most of the weird Safari bugs I was experiencing came from not having an actual video element "visible" on the page. Hope this helps!

[kamalk151]

The Issue still persists on iOS 14 release version.

Please can anyone provide solution for iOS 11 of 14 version? I am waiting for response.

[tsteinwen13]

The Issue still persists on iOS 14 release version.

Please can anyone provide solution for iOS 11 of 14 version? I am waiting for response.

I had the same problem till now.

the fix:

• append the video element to dom
• set video tag with css to: height: 0px, width: 0px and opacity: 0

this worked for me, all other fixes with playsinline and other stuff didn't worked.

[zhangchn]

The Issue still persists on iOS 14 release version. Please can anyone provide solution for iOS 11 of 14 version? I am waiting for response.

I had the same problem till now.

the fix:

* append the video element to dom

* set video tag with css to: height: 0px, width: 0px and opacity: 0

this worked for me, all other fixes with playsinline and other stuff didn't worked.

Another circumvent is to use a video element for playback, adding a hidden canvas for frame data copying, and an overlay canvas placing above the video element for feature displaying.

EDIT: This works on iOS 14.2 and Android. Actually, this approach showed much smoother video playback than drawing everything on a single canvas.

[mahasadhu]

@zhangchn do you have any code snippets to help further understand ?

[kamalk151]

Hi

Thanks for reply. But freezeing issue has been resolved. I have another issue that is related to resolution and scanning distance while scan the QR.

On Nov 13, 2020 3:30 PM, "Agus Mahasadhu" notifications@github.com wrote:

@zhangchn https://github.com/zhangchn do you have any code snippets to help further understand ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cozmo/jsQR/issues/185#issuecomment-726669216, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEBBGLCAZOPDNDI54A3W2STSPT7SDANCNFSM4O3KILHQ .

[zhangchn]

@zhangchn do you have any code snippets to help further understand ?

<div id="container">
  <canvas id="buffer"></canvas>
  <video id="video0" playsinline autoplay></video>
  <canvas id="overlay"></canvas>
</div>
<style>
canvas#buffer {
  display: none; /* for per-frame data copying */
}

canvas#overlay {
  position: absolute;
  top: 0;
  left: 0;
  width: 100%;
  height: 100%;
}

video#video0 {
  position: absolute;
  top: 0;
  left: 0;
  width: 100%;
  height: 100%;
}
</style>
<script>
import 'jsQR'
window.onload = () => {
const video = document.querySelector('video#video0')
const buffer = document.querySelector('canvas#buffer')
const overlay = document.querySelector('canvas#overlay')

const overlayCtx = overlay.getContext('2d')
const bufferCtx = buffer.getContext('2d')
let videoTick = null
let bounds = null

const tick = () => {
  if (video.readyState === video.HAVE_ENOUGH_DATA) {
    overlay.height = video.videoHeight
    overlay.width = video.videoWidth
    buffer.height = video.videoHeight
    buffer.width = video.videoWidth
    // hack the element size
    if (video.videoWidth / video.videoHeight < video.offsetWidth / video.offsetHeight) {
      overlay.style.height = video.offsetHeight + 'px'
      overlay.style.width = (video.videoWidth / video.videoHeight * video.offsetHeight) + 'px'
      overlay.style.left = (video.offsetWidth - video.videoWidth / video.videoHeight * video.offsetHeight) / 2 + 'px'
    } else if (video.videoWidth / video.videoHeight > video.offsetWidth / video.offsetHeight) {
      overlay.style.width = video.offsetWidth + 'px'
      overlay.style.height = (video.videoHeight / video.videoWidth * video.offsetWidth) + 'px'
      overlay.style.width = (video.offsetWidth - video.videoHeight / video.videoWidth * video.offsetWidth) / 2 + 'px'
    }
    bufferCtx.drawImage(video, 0, 0, buffer.width, buffer.height)
    const imageData = bufferCtx.getImageData(0, 0, buffer.width, buffer.height)

    // Call bounding box drawing here!
    if (bounds !== null) {
      overlayCtx.beginPath()
      overlayCtx.moveTo(bounds.topLeftCorner.x, bounds.topLeftCorner.y)
      overlayCtx.lineTo(bounds.topRightCorner.x, bounds.topRightCorner.y)
      overlayCtx.lineTo(bounds.bottomRightCorner.x, bounds.bottomRightCorner.y)
      overlayCtx.lineTo(bounds.bottomLeftCorner.x, bounds.bottomLeftCorner.y)
      overlayCtx.lineTo(bounds.topLeftCorner.x, bounds.topLeftCorner.y)

      overlayCtx.stroke()
    } else {
      overlayCtx.clearRect(0, 0, overlay.width, overlay.height)
    }
    try {
      code = jsQR(imageData.data, imageData.width, imageData.height, {
        inversionAttempts: 'dontInvert'
      })
      // Update detected QR boundary here
      bounds = code === null ? null : code.location
    } catch (err) {
      console.log(err)
    } 

  }
  // trampoline
  window.requestAnimationFrame(videoTick)
}

videoTick = tick

// set up video stream

if (typeof navigator.mediaDevices !== 'undefined') {
  navigator.mediaDevices.getUserMedia({
    audio: false,
    video: {
      facingMode: { exact: 'environment' },
    }
  }).then(stream => {
    video.srcObject = stream
    video.onloadedmetadata = () => {
      window.requestAnimationFrame(videoTick)
    }
  })
}
} // end of window.onload
</script>