search

jeeliz / jeelizFaceFilter

Javascript/WebGL lightweight face tracking library designed for augmented reality webcam filters. Features : multiple faces detection, rotation, mouth opening. Various integration examples are provided (Three.js, Babylon.js, FaceSwap, Canvas2D, CSS3D...).
https://jeeliz.com
Apache License 2.0
2.7k stars 534 forks source link

Live demos do not work. #218

Closed MariasStory closed 2 years ago

MariasStory commented 3 years ago

Your live demos use to work for me, and it was real fun to play with it. Now, none of your live demos work for me. I am in the latest chromuim. Is there something I am doing wrong?

There is no real error in the console. For example, the last console output for the following demo: https://jeeliz.com/demos/faceFilter/demos/threejs/dog_face/ Console output: INFO in JeelizResizer: bestCameraResolution = Object DevTools failed to load SourceMap: Could not load content for https://jeeliz.com/demos/faceFilter/libs/tween/v16_3_5/Tween.min.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE main.js:23 INFO: JEEFACEFILTERAPI IS READY three.js:856 THREE.WebGLRenderer 97 JeelizThreeHelper.js:398 INFO in JeelizThreeHelper - update_camera(): Estimated vertical video FoV is 35 JeelizThreeHelper.js:417 INFO in JeelizThreeHelper.update_camera(): camera vertical estimated FoV is 35 deg three.js:618 THREE.WebGLProgram: gl.getProgramInfoLog()

xavierjs commented 3 years ago

Provide info requested here: https://github.com/jeeliz/jeelizFaceFilter#compatibility (screenshots of webglreport) and also provide the log of https://jeeliz.com/demos/WebGLCoreLogger/index.html

and please provide more info: does it ask for your camera access? Did you enable webcam access near the URL bar?

MariasStory commented 3 years ago

Hi @xavierjs, Sorry for the delay with answer. Indeed, the demo works on windows, android and iOS. The problem is seen on linux. I did use few systems with linux chromium and the demos did not work. FF also did not work. The camera is on and it seems that something is going on in the background. Maybe someone can confirm this problem on linux?

xavierjs commented 3 years ago

Hi @MariasStory

I need more info. I also have Linux and it works like a charm, both on FF and Chrome. The compatibility also depends on graphic hardware, GPU drivers, specific Linux version, ... Not only the browser / OS.

Please provide:

Thank you

MariasStory commented 3 years ago

OK. Let it be:

============ INIT ContextFF ============
INFO in ContextFF: webglOptions =  {"antialias":true,"alpha":true,"preserveDrawingBuffer":true,"premultipliedAlpha":true,"stencil":false,"depth":true}
INFO in ContextFF: test if WebGL2 implementation is valid...
============ BEGIN SharedContext determine_floatRTTCapability() ============
INFO in SharedContext.test_RTT(): test RTT with glInternalPixelFormat = gl.RGBA32F, glPixelType = gl.FLOAT
INFO in ShadersFF.js: compile s41
INFO in ShadersFF.js: compile s40
  RTT success!
INFO in SharedContext.test_RTT(): test RTT with glInternalPixelFormat = gl.RGBA16F, glPixelType = gl.HALF_FLOAT
INFO in ShadersFF.js: compile s41
INFO in ShadersFF.js: compile s40
  RTT success!
============ END SharedContext determine_floatRTTCapability() ============
============= BEGIN TextureFilteringTester init() ============
INFO in ShadersFF.js: compile s40
INFO in TextureFilteringTester.test_mipmapping(): internalPixelFormat = gl.RGBA16F dataType = Uint16Array pixelType = gl.HALF_FLOAT isMipmap = true
INFO in TextureFilteringTester.test_mipmapping(): fail - read color is 0
INFO in TextureFilteringTester.test_mipmapping(): internalPixelFormat = gl.RGBA16F dataType = Float32Array pixelType = gl.FLOAT isMipmap = true
INFO in TextureFilteringTester.test_mipmapping(): success!
INFO in TextureFilteringTester: level = RGBAFloat16 - case 3 . isMipmap = true
============= END TextureFilteringTester init() ============
INFO in ContextFF: isValidWebGL2 =  true
INFO in ContextFF - init: We try to create a WebGL2 context
INFO in ContextFF - init: WebGL2 context has been initialized
INFO in SharedContext: enable_floatExtensions()
INFO in SharedContext: enable_halfFloatExtensions()
WARNING in SharedContext.enable_halfFloatExtensions(): OES_texture_half_float not found
WARNING in SharedContext.enable_halfFloatExtensions(): OES_texture_half_float_linear not found
INFO in SharedTexture: init()
============ BEGIN SharedContext determine_floatRTTCapability() ============
INFO in SharedContext.test_RTT(): test RTT with glInternalPixelFormat = gl.RGBA32F, glPixelType = gl.FLOAT
INFO in ShadersFF.js: compile s41
INFO in ShadersFF.js: compile s40
  RTT success!
INFO in SharedContext.test_RTT(): test RTT with glInternalPixelFormat = gl.RGBA16F, glPixelType = gl.HALF_FLOAT
INFO in ShadersFF.js: compile s41
INFO in ShadersFF.js: compile s40
  RTT success!
============ END SharedContext determine_floatRTTCapability() ============
INFO in SharedVBO : init()
INFO in SharedTexture: build random texture
INFO in ShadersFF.js: compile s0
INFO in ShadersFF.js: compile s1
INFO in ShadersFF.js: compile s2
INFO in ShadersFF.js: compile s3
INFO in ShadersFF.js: compile s4
INFO in ShadersFF.js: compile s5
INFO in ShadersFF.js: compile s6
INFO in ShadersFF.js: compile s7
INFO in ShadersFF.js: compile s8
INFO in ShadersFF.js: compile s9
INFO in ShadersFF.js: compile s10
INFO in ShadersFF.js: compile s11
INFO in ShadersFF.js: compile s12
INFO in ShadersFF.js: compile s13
INFO in ShadersFF.js: compile s14
INFO in ShadersFF.js: compile s15
INFO in ShadersFF.js: compile s16
INFO in ShadersFF.js: compile s17
INFO in ShadersFF.js: compile s18
INFO in ShadersFF.js: compile s19
INFO in ShadersFF.js: compile s20
INFO in ShadersFF.js: compile s21
INFO in ShadersFF.js: compile s22
INFO in ShadersFF.js: compile s23
INFO in ShadersFF.js: compile s24
INFO in ShadersFF.js: compile s25
INFO in ShadersFF.js: compile s26
INFO in ShadersFF.js: compile s27
INFO in ShadersFF.js: compile s28
INFO in ShadersFF.js: compile s29
INFO in ShadersFF.js: compile s30
INFO in ShadersFF.js: compile s31
INFO in ShadersFF.js: compile s32
INFO in ShadersFF.js: compile s33
INFO in ShadersFF.js: compile s34
INFO in ShadersFF.js: compile s35
INFO in ShadersFF.js: compile s36
INFO in ShadersFF.js: compile s37
============= BEGIN TextureFilteringTester init() ============
INFO in ShadersFF.js: compile s40
INFO in TextureFilteringTester.test_mipmapping(): internalPixelFormat = gl.RGBA16F dataType = Uint16Array pixelType = gl.HALF_FLOAT isMipmap = true
INFO in TextureFilteringTester.test_mipmapping(): fail - read color is 0
INFO in TextureFilteringTester.test_mipmapping(): internalPixelFormat = gl.RGBA16F dataType = Float32Array pixelType = gl.FLOAT isMipmap = true
INFO in TextureFilteringTester.test_mipmapping(): success!
INFO in TextureFilteringTester: level = RGBAFloat16 - case 3 . isMipmap = true
============= END TextureFilteringTester init() ============
INFO in WebGLCoreLogger: It should WORK YEAH!!!

image

image

image

xavierjs commented 3 years ago

Thank you for this information.

Are you sure you are on Linux? The user agent written on webglreport screenshots ( striped in red ) seems to be Windows NT Win64 x64 (on Linux it should be X11; Linux x86_6 or something like that).

According to the log the computing part should work. Maybe it is a problem with the video or the camera driver (it happens to me a few years ago with a Sony Xperia mobile phone, the bug was fixed by some system update). Can you see the video of your camera here: https://webrtc.github.io/samples/src/content/getusermedia/gum/ (you need to click on the Open Camera red button ).

Which FaceFilter demos are you trying? The hosted ones? What about this one: https://jeeliz.com/demos/faceFilter/demos/threejs/cube2cv/

MariasStory commented 3 years ago

Yes, it is linux. Please ignore the user string. The last link with cube2cv works.

xavierjs commented 3 years ago

If cube2cv works, then the lib work. If it does not work on a specific demo, I need to know which demo is not working and I also need the log of the web console on this demo, otherwise I cannot figure out what goes wrong.

chrsrns commented 3 years ago

I'm currently observing this on my Linux machine. Should I comment the diagnosis above here or should I open a new issue?

xavierjs commented 3 years ago

Hi @LifeWatcher123

I have re-open the issue. Please provide some information so that I can figure out what happens:

  1. On which demo does it happens?
  2. the browser, the version of the browser, the operating system, the version of the operating system, the device model and the GPU if it is a desktop computer,
  3. a screenshot of webglreport.com - WebGL1 (about your WebGL1 implementation),
  4. a screenshot of webglreport.com - WebGL2 (about your WebGL2 implementation),
  5. the log from the web console,
  6. The logs of https://jeeliz.com/demos/WebGLCoreLogger/index.html
  7. Does https://jeeliz.com/demos/faceFilter/demos/threejs/cube2cv/ work? Does it ask for camera access?

Sometimes another app using the camera can block the browser from getting the video stream.

Thanx, Xavier

chrsrns commented 3 years ago

Hi @LifeWatcher123

I have re-open the issue. Please provide some information so that I can figure out what happens:

  1. On which demo does it happens?
  2. the browser, the version of the browser, the operating system, the version of the operating system, the device model and the GPU if it is a desktop computer,
  3. a screenshot of webglreport.com - WebGL1 (about your WebGL1 implementation),
  4. a screenshot of webglreport.com - WebGL2 (about your WebGL2 implementation),
  5. the log from the web console,
  6. The logs of https://jeeliz.com/demos/WebGLCoreLogger/index.html
  7. Does https://jeeliz.com/demos/faceFilter/demos/threejs/cube2cv/ work? Does it ask for camera access?

Sometimes another app using the camera can block the browser from getting the video stream.

Thanx, Xavier

I've tested it on the following filters:

Machine Info:

Logs and WebGL Infos:

chrsrns commented 3 years ago

From what I've read on the issue, I think its important that I disclose that I am using Droidcam's virtual camera. The filters work on the iOS device, but not when used as a virtual camera.

xavierjs commented 3 years ago

Hi @LifeWatcher123

If the cube work works then it is not a WebGL / Camera issue.

Can you provide a screenshot of chrome://gpu (it should work on Vivaldi Browser since it is based on Chromium) ? Maybe there are some major GPU issues and fallbacks to software rendering. It can also comes from a specific GPU issue, which won't be astonishing since AMD drivers are not so great under Linux (I am also a Linux user and I always choose Nvidia/Intel for this reason).

What's about this demo: https://jeeliz.com/demos/faceFilter/demos/threejs/cube/ Does it work better than the previous cube one?

What is your current CPU model? I need to check the benchmark of the integrated GPU to see if it can just be a computing power issue.

Thanx, Xavier

chrsrns commented 3 years ago

If the cube work works then it is not a WebGL / Camera issue.

The cube was going on random spots on the video feed. But it does render, if that what determines whether WebGL works.

What's about this demo: https://jeeliz.com/demos/faceFilter/demos/threejs/cube/ Does it work better than the previous cube one?

No, unfortunately. It doesn't render anything above the video feed.

What is your current CPU model?

My CPU model is AMD A8-7650K Radeon R7 4C+6G

Here's the copy of the output from chrome://gpu: Unfortunately, I my screen capture browser extension can't capture this one. Here's a portion of the output for now while I find another tool for the screenshot (off topic: do you know a Linux screenshot utility that does a scrolling screenshot?): image

xavierjs commented 3 years ago

Thanx for this information @LifeWatcher123 . Your GPU is good and hardware acceleration is fine. The problem should be elsewhere... Can you test:

I hope at least one of the 2 will work... Thank you by advance

chrsrns commented 3 years ago

Thanx for this information @LifeWatcher123 . Your GPU is good and hardware acceleration is fine. The problem should be elsewhere... Can you test:

I hope at least one of the 2 will work... Thank you by advance

No! Both works splendidly!

xavierjs commented 3 years ago

Thanx @LifeWatcher123 The update on the master branch is coming in the next weeks (to celebrate the 2000 stars), with upgraded neural networks too (they are still training).

chrsrns commented 3 years ago

Neat! Can't wait.

Will the demos get updated automatically after that or are manual compilations necessary?

xavierjs commented 3 years ago

The static JavaScript demos will be upgraded (with the new FaceFilter and neural net). However, those running on a deprecated version of THREE.js (because using the legacy JSON format instead of GLTF) will still run on this old THREE.js.

The NPM package will be upgraded too so the react demo will be upgraded (new FaceFilter and new neural network)

xavierjs commented 2 years ago

I close the issue since it seems revolved, feel free to reopen it if the bug still occurs.