Frequent call of binarize method then run in webworker

[SAprelov]

It is very misterious, bt...

My guess its GC. Any idea how to avoid this?

I found it in Chrome 78. It works in Firefox and Safari.

[cabdesigns]

We've also stumbled into this around the same time. It's replicated by having the QR code process in a web worker as part of a video stream - and leaving it alone for a while.

Usually on a low powered device, the QR code would be processed in around 100ms. After a while however, it seems to take a drastic degradation and every QR code processed takes 3s in the web worker.

There are a lot of binarize calls, and they are slow. I'm not sure what the answer is right now.

[cabdesigns]

I've tried a lot of things to work around this and the only solution I've found is to strip out the web worker entirely. Instead I'm debouncing the jsqr call every 100ms on the main thread and that's actually turning out to be a lot more performant.

Previously, it would start fine but eventually the web worker thread would max out CPU, making the jsqr a lot slower to complete and every call there after would also be slow.

I'm happy enough with this solution for now. Hope it also works for you @SAprelov.

[FPWombot]

I've tried a lot of things to work around this and the only solution I've found is to strip out the web worker entirely. Instead I'm debouncing the jsqr call every 100ms on the main thread and that's actually turning out to be a lot more performant.

Previously, it would start fine but eventually the web worker thread would max out CPU, making the jsqr a lot slower to complete and every call there after would also be slow.

I'm happy enough with this solution for now. Hope it also works for you @SAprelov.

Do you know of a way to "destroy" the scanner instance when not in use?

[cozmo]

I think there's a lot of confusion here, which is potentially making this harder to detangle. It's possible there's actually a bug too, but lets just clear up some of the general questions first.

• There's no "instance" of jsQR, nor is it asynchronous etc - jsQR is just an (resource intensive) function call, and so when processing video you can choose to call it or not. For example @FPWombot if you want to stop scanning once you've detected a QR code you could just do something like this

function tick() {
  // get the image data and scan it for a QR code
  var imageData = canvas.getImageData(0, 0, canvasElement.width, canvasElement.height);
  var code = jsQR(imageData.data, imageData.width, imageData.height);

  if (code) {
    // If we found a code log it, but _don't_ call tick anymore, since we're done scanning
    console.log("Found code", code);
  } else {
    // If we didn't find a code then run tick on the next frame
    requestAnimationFrame(tick);
  }
}

// run tick on the next frame
requestAnimationFrame(tick);

This would cause tick to run every animation frame, until a code is found, at which point nothing else will run. This is what you mean by "destroying" the scanner, but like say since there's no instance or anything to "destroy" this is closer to what it would actually look like.

That said, you may still run into performance issues if calling jsQR takes longer than 1 frame - Since JS is single threaded when you're running jsQR on the main thread (like that example does) the video/other computations on the main thread will stutter if jsQR is taking more than 1 frame of time to run. That's why in performance sensitive situations people often offload the processing/resource intensive work to a webworker. I've worked on production setups with jsQR that did this, and haven't seen the behavior described above.

A thing to keep in mind is if the jsQR call takes more than 1 frame to process, even if you move it to a web worker you can't just send every frame of data to it, since they'll then backlog. My guess as to what's going on above is that you're not waiting for the web worker to finish processing a frame before sending the next frame of data.

The setup I've seen run without any issue looks something like:

// worker.js

// Respond to message from main thread
self.addEventListener("message", message => {
  const { width, height, data } = message.data;
  // Respond with the result of runnning jsQR on this frame
  self.postMessage(jsQR(data, width, height));
});

// index.js (main thread)
var worker = new Worker(`worker.js`)
var workerReady = true; // is the worker ready to process a frame

function tick() {
  // get the image data
  var imageData = canvas.getImageData(0, 0, canvasElement.width, canvasElement.height);

  // If the worker is ready to process a frame then send it one
  if (workerReady) {
    worker.postMessage(imageData, [imageData.data.buffer]);
    // worker is not ready to accept another frame until it finishes processing this one.
    workerReady = false;
  }
}

// We got a message back from the worker
worker.onmessage = function(m) {
  workerReady = true; // Since it's done processing the last frame it's ready to accept another one.
  if (m) { // If we got a code back log it. 
    console.log("Found code!", m);
  }
}

// run tick on the next frame
requestAnimationFrame(tick);

This means that the worker is only ever processing 1 frame, so no queue/backlog. The flip side of this is is if it takes N frames to run jsQR then jsQR will only ever be run every N frames. If you want to ensure you scan every frame for some reason (haven't needed to do this) you'd want to spin up a pool of workers.

Generally the takeaway here is jsQR can take more than 1 frame to run, so running it on every frame is going to cause performance issues, BUT if you avoid that problem and offload the work to a web worker the performance issues should be resolved (at least in my experience). LMK if that helps/if this issue persists, happy to help debug more, as this is a problem I've worked on a fair amount.

[SAprelov]

@cozmo thank you for detailed explanation. Bt.. I have already avoid throttling. Yes, my worker gets only one frame and ignores next frames if still busy. I didnt change my code for months and it works like a charm until Chome 78. I have buisiness app and over hundred clients and they report same problem at one moment after Chrome upgrade. I have tried to find any information about change of chrome webworker behavior but no luck.

[FPWombot]

@cozmo thankyou for such a quick reply. Ok this makes alot of more sense with the tick call. I guess what I meant was how to kill the video stream. For some context. I have a site for a non profit organization that is for race check-in. So we scan their QR code and enter their bib number. This is done on mobile devices so batter life is a concern. Is it possibly to kill the video feed when not in use without causing the jsqr to have any issues.

Also, I apologize as I am not a pro like you, but do you happen to have any reference or tutorial I could look at to learn about using a web worker as I am not familiar with the topic. And would using a web worker use less CPU ok the device or at least cause less lagg on the check-in page?

Thank you so much for your time.

[cozmo]

@SAprelov Ah that's really interesting - promising that you're already doing the queuing/blocking. As far as I know I've not heard issues on any versions of Chrome - Is it possible to get a small repro case (simple html+js+worker.js on https://gist.github.com or such)? I'd love to dig in a bit.

@FPWombot I'm not an expert in video streams unfortunately. Enough so that I explicitly have to call out that I'm happy to help debug jsQR issues, but will be a lot less helpful with general video streaming questions (since jsQR is designed to run separately from the image source).

That said I found this - https://stackoverflow.com/questions/11642926/stop-close-webcam-which-is-opened-by-navigator-getusermedia and it seems like it could be roughly what you're looking for?

As for the WebWorkers stuff I think my rough example above is almost workable, but if you want a more general introduction maybe something like this article is a good interaction - https://blog.teamtreehouse.com/using-web-workers-to-speed-up-your-javascript-applications

[SAprelov]

Can`t reproduce this bug after Chrome update). Current version is 79.0.3945.130. Need more time)

[leaderbryce]

Hello everyone ! I tried to set the worker solution from @cozmo , it works great, but for some reason, the drawCode function, that is supposed to draw the detected code in green is not working anymore (works great with the no worker solution). Any hint about this ?

Thank you so much Brice