jeeliz / jeelizFaceFilter

Javascript/WebGL lightweight face tracking library designed for augmented reality webcam filters. Features : multiple faces detection, rotation, mouth opening. Various integration examples are provided (Three.js, Babylon.js, FaceSwap, Canvas2D, CSS3D...).
https://jeeliz.com
Apache License 2.0
2.68k stars 533 forks source link

Any idea why it doesn't work on Android 6.0 or Android 7.0 ? With the latest version of Chrome #63

Closed alexbudin closed 5 years ago

alexbudin commented 5 years ago

Tried the ThreeJS Boilerplate and the CSS3D one and none of them work on Android 6 or 7 with the latest version of Chrome.

Any idea why? Or what we can do to make it work or at least on Android 7?

Thank you.

alexbudin commented 5 years ago

All the below is for a Samsung Galaxy S5 - Chrome latest version (70.0.3538.80) - Android 6.0.1

I get this error: AN ERROR HAPPENS. ERR = GL_INCOMPATIBLE

And these warnings: (index):1 [.WebGL-0xae0ba600]GL ERROR :GL_INVALID_OPERATION : glGenerateMipmap: Can not generate mips (index):1 [.WebGL-0xae0ba600]RENDER WARNING: texture bound to texture unit 0 is not renderable. It maybe non-power-of-2 and have incompatible texture filtering. (index):1 [.WebGL-0xae0ba600]GL ERROR :GL_INVALID_OPERATION : glGenerateMipmap: Can not generate mips 3(index):1 [.WebGL-0xae0ba600]RENDER WARNING: texture bound to texture unit 0 is not renderable. It maybe non-power-of-2 and have incompatible texture filtering.

Doing the test here: https://jeeliz.com/tests/libGetUserMedia/

I got the following:

INFO in lib_getUserMedia() - get() : isMobile but not iOS lib_getUserMedia.js:43 INFO in lib_getUserMedia() - get() : constraints = {"video":{"facingMode":{"ideal":"user"},"width":{"min":480,"max":1280,"ideal":600},"height":{"min":480,"max":1280,"ideal":800}},"audio":false} lib_getUserMedia.js:30 INFO in lib_getUserMedia - get_raw(): videoStream got lib_getUserMedia.js:34 INFO in lib_getUserMedia - get_raw() : video.onloadedmetadata dispatched lib_getUserMedia.js:34 INFO in lib_getUserMedia - get_raw() : playPromise accepted lib_getUserMedia.js:29 INFO in lib_getUserMedia - get_raw(): callbackSuccess called with constraints= lib_getUserMedia.js:29 {"facingMode":{"ideal":"user"},"width":{"min":480,"max":1280,"ideal":600},"height":{"min":480,"max":1280,"ideal":800}} (index):26 callbackSuccess (index):27 resolution : 0 0 (index):28 resolution2 : 800 600

image

And it doesnt seem to support WebGL2:

image

alexbudin commented 5 years ago

I am now testing on a Android 7 with latest chrome and something weird is happening. It asks for access to the camera, but nothing shows up (i tried with tons of basic webcam demos)

This is what i get when i go here: https://jeeliz.com/tests/libGetUserMedia/

INFO in lib_getUserMedia() - get() : isMobile but not iOS
lib_getUserMedia.js:43 INFO in lib_getUserMedia() - get() : constraints = {"video":{"facingMode":{"ideal":"user"},"width":{"min":480,"max":1280,"ideal":600},"height":{"min":480,"max":1280,"ideal":800}},"audio":false}
lib_getUserMedia.js:30 INFO in lib_getUserMedia - get_raw(): videoStream got

Then after about 10 seconds, this appears:

INFO in lib_getUserMedia() - get() : isMobile but not iOS;
lib_getUserMedia.js:43 INFO in lib_getUserMedia() - get() : constraints = {"video":{"facingMode":{"ideal":"user"},"width":{"min":480,"max":1280,"ideal":600},"height":{"min":480,"max":1280,"ideal":800}},"audio":false}
lib_getUserMedia.js:30 INFO in lib_getUserMedia - get_raw(): videoStream got
lib_getUserMedia.js:34 INFO in lib_getUserMedia - get_raw() : video.onloadedmetadata dispatched
lib_getUserMedia.js:34 INFO in lib_getUserMedia - get_raw() : playPromise accepted
lib_getUserMedia.js:43 WARNING in lib_getUserMedia() - get() : cannot request video with this constraints. Downgrade constraints. err= VIDEO_NOTSTARTED
lib_getUserMedia.js:33 INFO in callSuccessIfPlaying(): video.currentTime= 0
lib_getUserMedia.js:30 INFO in lib_getUserMedia - get_raw(): videoStream got

BUT nothing shows up on the browser, just an empty page. Briefly i see a black rectangle that dissapears after a few ms.

This is the WEBGL info:

image

image

image

image

image

xavierjs commented 5 years ago

Hi,

Thank you very much for these detailed debug informations. The problem comes from the graphic drivers, not the OS or browser.

FaceFilter first try to get a WebGL2 context because it does not require any extension if WebGL2 is implemented (if it is done correctly). Here, he cannot get the WebGL2 context.

So then he got a WebGL1 context. But with WebGL1 we absolutely need some extensions (which have been included into the WebGL 2 standard) in order to handle FLOAT or HALF_FLOAT textures.

But these extensions are not here (there is no OES_TEXTURE_FLOAT or OES_TEXTURE_HALF_FLOAT).

We stil do some real tests in order to check if it works even without the extensions (that's why there are some warnings in the JS console) but it is not possible to handle float/half float textures.

If you open chrome://gpu the information relative to the GPU driver. On Android the graphic drivers are usually updated with system updates.

alexbudin commented 5 years ago

And what about the second phone. The Android 7. It has WebGL2 support and for some reason the webcam does not show up at all and there are no errors ?

Surely there must be a bug or something or maybe im missing something.

Thank you.

xavierjs commented 5 years ago

The getUserMedia helper you tested is deprecated. There was some mistakes (on some Android devices the delay to get the video stream can be quite long, the old getUserMedia helper did not wait long enough), can you try the new one: https://jeeliz.com/tests/libGetUserMedia2/

What are these 2 phones exactly ?

i am still quite disappointed about the quality of WebGL2 implementation. Unfortunately, after a WebGL2 initialization failed attempt, it is not possible to remove the WebGL2 context from the canvas and to retry with a WebGL1 context.

And I hope that this is temporary bugs which will be solved when more and more WebGL2 applications will require full WebGL2 capability.

We tested our apps with Samsung galaxy S7, S8, A5, Iphone 6 and X, Ipad pro, Sony Xperia Zplay, and we tested it promptly with Motorolla Zplay, One plus, Huawei P9. But there are so many configs... :(

alexbudin commented 5 years ago

The phones are a Samsung Galaxy S5 (Android 6) - which does not have WebGL2 so i kind of understand this wont work, and the other phone is a Huawei P10 (vtr-L29) with Android 7 and WebGL2 support.

I tested this: https://jeeliz.com/tests/libGetUserMedia2/ on the Huawei P10 with no success, i get the same blank page :(

xavierjs commented 5 years ago

For the Galaxy S5 I think it is a bit too old for this kind of applications. I already tested with a S4 and it does not work.

For the Huawei it is really strange, it should work. I think it is a getUserMedia problem. Does the official getUserMedia tests work ? Like this one: https://simpl.info/getusermedia/ or this one: https://webrtc.github.io/samples/src/content/getusermedia/resolution/

alexbudin commented 5 years ago

Added this as a bug on Chromium. I tested the links you gave me on Chrome and Chrome Beta and Canary with no success, BUT, it works on Firefox. So its deff a chrome problem.

https://bugs.chromium.org/p/chromium/issues/detail?id=906574&can=2&start=0&num=100&q=&colspec=ID%20Pri%20M%20Stars%20ReleaseBlock%20Component%20Status%20Owner%20Summary%20OS%20Modified&groupby=&sort=

xavierjs commented 5 years ago

Thank you very much.

I had a similar problem 2 years ago with a Sony Xperia, the video got on Chrome was black and 2 pixels wide (exactly this issue: https://talk.sonymobile.com/t5/Android-development/Can-t-start-camera-with-HTML5-camera-API-getusermedia/td-p/1167372#gref). It has been corrected.

Chromium devs are very reactive, so if the problem comes from Chrome I am confident it will be solved soon. But if it comes from Huawei or the webcam driver provided by the manufacturer it may take longer ...

alexbudin commented 5 years ago

Should be Chrome, as it works on Firefox :)

xavierjs commented 5 years ago

For my problem it was the same: it worked on Firefox but not on Chrome. But it was because Firefox had added a workaround or something like that, but the real problem was not due to the browser. I think browser makers are very prone to add fixes to cope with driver or OS mistakes, when I take a look at chrome://gpu I am always afraid to see the list of Problems detected...