search

serratus / quaggaJS

An advanced barcode-scanner written in JavaScript
https://serratus.github.io/quaggaJS/
MIT License
5.06k stars 978 forks source link

Quagga with Cordova Android app #94

Closed ant-1 closed 4 years ago

ant-1 commented 8 years ago

Hello !

I'm trying to include the quagga barcode scanner in a Cordova Android webapp and I have some troubles.

If I run my webapp in a browser on my pc, everything seems to be fine, my webcam is activated and I can display the livestream (except some errors in the browser log) But if I run my app on my Android phone as an Android webapp (wrapped in java code), I can't see anything.

The only difference between browser and Android version is the DOM video element created:

In Chrome browser: <div id="barcodeScanner" style="width: 300px; height: 300px"><video autoplay="true" preload="auto" src="blob:http%3A//localhost%3A3000/916803b0-94b7-4222-a987-da251a332e87"></video><canvas class="drawingBuffer" width="640" height="480"></canvas><br clear="all"></div>

In Android app: <div id="barcodeScanner" style="width: 300px; height: 300px"><video autoplay="true" preload="auto" src="blob:file%3A///0bf84747-caac-41e8-97b9-6ef224df8692"></video></div>

No problem with the getUserMedia API, it is available on my Android app. And I also add access permissions to the camera of my phone

Any suggestion to investigate on my problem ?

Thank you

[Edit] Also tried to load in my android "web" application the camera_example.html. It shows nothing on the screen and I get this html: <body><video src="blob:file%3A///082ae35f-d89b-4c37-9782-d57562d80084"></video><canvas width="2" height="2"></canvas></body>

nicolas-rempulski commented 8 years ago

Hi,

I hijack your issue to try to answer it and add another question about Cordova. We use quaggaJS in Cordova environment too and had some issues making it works.

I think your issue is about hybrid application native part. Even if your WebView is able to handle HTML5 and WebRTC, the native part may not allow the application to have access to the camera. If you try to list available MediaSource from your Cordova application with : 

MediaStreamTrack.getSources(function(sources){
        console.log(sources);
      });

You should see only one source, which is audio only.

To have access to Camera from your application, and thus let quaggaJS auto-start the stream, you should ask android for permission to access Camera. You have to add to your AndroidManifest.xml

<uses-permission android:name="android.permission.CAMERA" />

Now, media listing should return 3 streams : audio, environment (back camera), user(not sure on the name on this one, but it's the front camera). quaggaJS should now be able to start.

This permission shouldn't be erased by Cordova upon rebuild, but it will be removed on platform update or add. There is surely a way to add this permission automatically from Cordova config.xml or via Cordova hooks, but we do it only manually for now.

As for our experience / question :

We made it works in our Cordova app but we hit a bottleneck with performance issues. We tested this a few months ago on mobile devices, the video stream was lagging due to the high processing needed by quagga. We tried several things with configuration, removing all location and debugs informations, halfsampling, play with workers number, use our own canvas and send Image as Imagestream to quagga... We also tried to go inside quagga to customize some process (that didn't end well ;) )

When we open our own stream via WebRTC, the stream is smooth, I would say "Camera application" like. As soon as we quagga.start() the stream start to be laggy (3-4 FPS). We also tried to use quagga directly on device browser, to ensure it was not a Cordova problem, and we hit the same issue.

I see that there is now, in master, a frequency configuration allowing us to set the analysis number per second. It is a great solution and workaround for mobile environment and I would like to try it but I have an issue with workers postMessage function that I never encountered nor that I understand : 

Uncaught DataCloneError: Failed to execute 'postMessage' on 'Worker': An object could not be cloned.

thrown by 

        workerThread.worker.postMessage({
            cmd: 'init',
            size: { x: _inputStream.getWidth(), y: _inputStream.getHeight() },
            imageData: workerThread.imageData,
            config: _config
        }, [workerThread.imageData.buffer]);

I read, on threads about workers, that this error can be generated if a postMessage try to pass a parameter which is an object and this object has functions. I cannot figure which parameter it could be in this case or how I could work around it. Or even if it is really the problem here.

Did anyone had this issue on mobile environment or at all ?

Thanks !

nicolas-rempulski commented 8 years ago

I got it, I will open another issue but the problem was that I specified a "target" in configuration.

I think the error was thrown because HTMLElement specified in "target" is a complex object that the browser can't clone. If I remove it, and thus the _config passed in postMessage doesn't contain it, it works like a charm.

ant-1 commented 8 years ago

Thanks Nicolas for the answer.

Actually, as I said in my first post, I've ever added the right user permissions in my android.manifest to access the camera. And the result remain the same, I do not see any streams.

I've made a simple cordova app to test the stream here The index.html file is exactly the same that we can find provided in the Quagga examples folder

Any idea to make it work ?

fatbearLTG commented 8 years ago

Have you tried your project in a android chrome browser? Will it show the stream OK?

suavelizard commented 6 years ago

@ant-1 did you manage to resolve your issue?

tonno commented 6 years ago

I have this exact same problem. I keep getting a PermissionDeniedError in Quagga.init(). I have tried setting all permissions in Androidmanifest.xml, and also requesting the permissions at runtime, but no luck.

I suspect the reason is that the Chrome browser in recent Androids just considers file:///android_asset/www insecure and refuses to let it use the video stream. Even though file:// is supposed to be "potentially secure" and should be allowed to use the video stream.

I am getting desperate. My code works if I run it from HTTPS server, but when I package it into an apk and install it then it doesn't work.

Is there any way to get around that PermissionDeniedError?

Cheers Tommi Joutsiniemi

tonno commented 6 years ago

Maybe this is the solution: http://www.flapjacksandcode.com/blog/2015/2/17/android-webviews-getusermedia-putting-it-all-together ???

ericblade commented 6 years ago

Anyone who's commented on this found a specific answer, or know if this problem is still valid?

mathiasmoeller commented 4 years ago

Anyone who's commented on this found a specific answer, or know if this problem is still valid?

For me this is still a valid problem

ericblade commented 4 years ago

If someone can give me a quick "how to ship quagga and cordova to a device" tutorial, i can give it a spin and see what i can find...

mathiasmoeller commented 4 years ago

The main reason it does not work with iOS is due to a problem in Phonegap regarding iOS. See here: https://github.com/phonegap/phonegap-plugin-media-stream#quirks Even with access to the MediaStream you can not attach it to a <video> tag. Which Quagga does. As far as I understand it, it is simply not possible to use Quagga (or any other thing that displays a camera video in a <video> tag) with Phonegap on iOS.

ericblade commented 4 years ago

hmm. that's weird how that works in iOS, if I had a useful iOS device, i could maybe figure out a way to structure something to work with it.. i have no concrete idea why it doesn't work on Android, though, but my first guess is "permissions", and probably needing to implement the call in Android that actually allows the WebView to access the camera. (the device will ask for permissions correctly when you install it, but Chrome quite possibly still needs to be given permission on it's own)

chrisrodriguezkinesso commented 4 years ago

how is this still open, anyone got this working with Cordova?

ericblade commented 4 years ago

Well, I've forked the library over here https://github.com/ericblade/quagga2 ... and I ship it to Android directly to the browser from a webserver. At some point, I would like to be able to load my app directly to device, but it's not been at all a priority for me. If someone wants to try it with my repo, and provide a framework for loading it to devices, I can try to have a look at it.

chrisrodriguezkinesso commented 4 years ago

I got it working actually, it's related to permissions, just install Cordova permissions plugin, and wrap with permission to enable the camera.

let permissions = cordova.plugins.permissions; permissions.checkPermission(permissions.CAMERA, (res) => { if (!res.hasPermission) { permissions.requestPermission(permissions.CAMERA, open());

ericblade commented 4 years ago

... sounds like my guess was right. I guess having some experience with implementing Chrome into embedded systems is useful sometimes :-D