faceres model returns null values for humangl backend

[shurshilov]

Issue Description After update Human to 2.9.3 and tf 3.20. face embedding always Nan

Steps to Reproduce Config Human as

Expected Behavior All work good and const found = result.face[0].embedding return 1024 array with values Environment Chrome 104.0.5112.102 (64 бит)

• Human library version?

• 2.9.3

• Built-in demo or custom code?

• 

• Type of module used (e.g. js, esm, esm-nobundle)?

• 'https://cdnjs.cloudflare.com/ajax/libs/human/2.9.3/human.js',

• TensorFlow/JS version (if not using bundled module)?

• 'https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@3.20.0/dist/tf.min.js',

• Browser or NodeJS and version (e.g. NodeJS 14.15 or Chrome 89)?

• Chrome 104.0.5112.102 (64 бит)

• OS and Hardware platform (e.g. Windows 10, Ubuntu Linux on x64, Android 10)?

• Windows 10

• Packager (if any) (e.g, webpack, rollup, parcel, esbuild, etc.)?

• not

• Framework (if any) (e.g. React, NextJS, etc.)?

• pure js

Diagnostics

• Check out any applicable diagnostic steps

Additional

• For installation or startup issues include your package.json
• For usage issues, it is recommended to post your code as gist
• For general questions, create a discussion topic

[vladmandic]

can you try using 2.9.4 that is on github (its not yet published on npmjs and not mirrored on any cdn)?

i've tried to reproduce using your configuration and i cannot:

const fs = require('fs');
const tf = require('@tensorflow/tfjs-node');
const Human = require('../../dist/human.node.js');

const humanConfig = {
  filter: { equalization: true },
  face: {
    enabled: true,
    detector: { rotation: true, return: true, cropFactor: 1.6, mask: false },
    description: { enabled: true }, emotion: { enabled: true },
    iris: { enabled: true }, anispoof: { enabled: true }, liveness: { enabled: true } },
};

const human = new Human.Human(humanConfig);
const buffer = fs.readFileSync('samples/in/ai-body.jpg');
const tensor = human.tf.node.decodeImage(buffer);
console.log({ human: human.version, tfjs: tf.version_core, resolution: tensor.shape });
human.detect(tensor).then((result) => console.log(result.face[0].embedding));

{ human: '2.9.4', tfjs: '3.20.0', resolution: [ 1200, 1200, 3 ] }
[
     0.08139020204544067,                     0,   0.11821223050355911,
      0.3587932288646698,   0.04746624082326889,   0.17449817061424255,
     0.16748841106891632,  0.021941592916846275,                     0,
                       0,    0.8505107164382935,    0.6311725974082947,
     0.38210225105285645,      1.41637122631073, 0.0022489684633910656,
     0.02798292599618435,    0.2757798731327057,   0.03239351883530617,
    0.007485136855393648,                     0,                     0,
      0.3470226228237152, 0.0038550239987671375,   0.21544772386550903,
  0.00045395689085125923,    0.0379939042031765,   0.19062575697898865,
                       0,   0.09399671107530594,   0.19237107038497925,
  ...
]

also, in the future can you post config and/or code as text?
it makes it much easier to copy&paste than to re-type everything from an image screenshot?

on a separate note, why are you loading both human.js and tf.min.js?
tfjs is already embedded in human unless you're explicitly using no-bundle version of human

[shurshilov]

About snapshot sorry( I do some test and this bug start from 2.6.4 2.6.3 work good. My version was 2.5.6 and I updated to 2.9.3 I try 2.8.1 2.7.1. 2.9.1 and all version with this bug only 2.6.3 and lower work good Look like from 2.6.4 break detect (canvas)

[vladmandic]

i just tried using 2.9.1, 2.8.1, 2.7,1 and i cannot reproduce - seems to be working fine.
i couldn't test 2.6.1 anymore as its not compatible with current versions of nodejs and i don't want to downgrade everything on my system.

just to check - the rest of result is ok, its just embedding that returns NaN array?

also, i've checked diff between 2.6.3 and 2.6.4 and that was one of the smallest releases,
it was a bugfix for loading of wasm binaries.

[shurshilov]

Yes all object good

Also I found that gender FALSE

[shurshilov]

I set backend: 'webgl' and it work good on all versions and 2.9.1 too But now all faces 100% similiraty(((but descriptors different const found = result.face[0].embedding const similarity = this.parent.human.similarity(Array.from(this.descriptor_ids[i]),found); console.log(found, 'found') console.log(Array.from(this.descriptor_ids[i]), 'db') console.log(face ${i} is ${100 * similarity}% simmilar); // Found employee console.log(100 * similarity)

[vladmandic]

I set backend: 'webgl' and it work good on all versions and 2.9.1 too

which backend was it before when it was not working?
what occured to me is that model signatures changed a while ago as well, so when you're updating human library, you should also update your locally stored models (stuff in your modelBasePath)

please check your browser console output for any possible errors - most likely there is an issue loading model or something like that.

i just added automatic test for human.similarity() and human.match() for all browser backends and its working as expected:

test similarity/wasm { match: {index: 0, distance: 2331.089495461392, similarity: 0.5286440393506266}, similarity: 0.5286440393506266 }
test similarity/humangl { match: {index: 0, distance: 2331.0913441264966, similarity: 0.5286437202722033}, similarity: 0.5286437202722033 }
test similarity/webgl { match: {index: 0, distance: 2331.0913441264966, similarity: 0.5286437202722033}, similarity: 0.5286437202722033 }
test similarity/webgpu { match: {index: 0, distance: 2331.092204806524, similarity: 0.5286435717194199}, similarity: 0.5286435717194199 }

[shurshilov]

I set backend: 'webgl' and it work good on all versions and 2.9.1 too

which backend was it before when it was not working? what occured to me is that model signatures changed a while ago as well, so when you're updating human library, you should also update your locally stored models (stuff in your modelBasePath)

please check your browser console output for any possible errors - most likely there is an issue loading model or something like that.

i just added automatic test for human.similarity() and human.match() for all browser backends and its working as expected:

test similarity/wasm { match: {index: 0, distance: 2331.089495461392, similarity: 0.5286440393506266}, similarity: 0.5286440393506266 }
test similarity/humangl { match: {index: 0, distance: 2331.0913441264966, similarity: 0.5286437202722033}, similarity: 0.5286437202722033 }
test similarity/webgl { match: {index: 0, distance: 2331.0913441264966, similarity: 0.5286437202722033}, similarity: 0.5286437202722033 }
test similarity/webgpu { match: {index: 0, distance: 2331.092204806524, similarity: 0.5286435717194199}, similarity: 0.5286435717194199 }

Hmm streange, so may be i share access to my test website with error? Yes about models I also noticed this. And after 2.6.1 a clear indexDB every time and replace model folder if change human.js version, but error still

[vladmandic]

so may be i share access to my test website with error

sure

[shurshilov]

so may be i share access to my test website with error

sure

https://eurodoo.com/web#action=543&cids=1&menu_id=374 login demo1 pass demo1 click check in will open face rec dialog

[vladmandic]

• this is a valid descriptor array: result.face[0].embedding
• this is NOT a valid descriptor array: Array.from(this.descriptor_ids[i])

so you get good results from human, but they get stored incorrectly?
and i have no idea whats going on here!?

self.descriptor_ids=[];
for(var f32base64 of data.descriptor_ids) {
  self.descriptor_ids.push(new Float32Array(new Uint8Array([...atob(f32base64)].map(c=>c.charCodeAt(0))).buffer))
}

[shurshilov]

Its converter JS Array 1024 to base64 for save in DB. Just Serialization but its really not easy, And deserialization proccess from base64 to array 1024. And reason after upadet that result.face[0].embedding return 1024 Nan it can check in console i print it as "found" text

[vladmandic]

And that "found" contains invalid array.

So I'm assuming your serialization is where things go bad. The type of results.face.embedding changed since 2.6.3 - that was 3 major versions ago (things never change in minor version).

[shurshilov]

No, serialization its jus additional feature, but results.face.embedding return 1024 array with Nan and its not depend serialization after it. It should contain Array with numbers...? How i can get old Array 1024 results.face.embedding ? I just do const found = result.face[0].embedding and I'm expecting an array with numbers. But I get an array with Nan.

[vladmandic]

from YOUR DEMO:

const found=result.face[0].embedding
console.log(found,'found')

(1024) [0.2984731197357178, 0.2595631778240204, 0.3963339328765869, 0, 0, 0, 0.23985564708709717, 0.7768914103507996, 1.177507758140564, 0.7845110893249512, 0, 0.02752731181681156, 0, 0.09198596328496933, 0.4361388087272644, 0, 0.18945324420928955, 0.0012000531423836946, 0.005083911120891571, 0, 0.36032697558403015, 0.035741083323955536, 0.07470899820327759, 0, 0, 0, 1.1794145107269287, 0.9509024024009705, 0.1838228404521942, 0.1625743806362152, 0.012997782789170742, 0.15879793465137482, 0, 1.0624027252197266, 1.2277904748916626, 0, 0.002714419737458229, 0.014698048122227192, 0, 0, 0.11073991656303406, 0, 0.21221040189266205, 0, 0, 0, 0, 0.4801158308982849, 0.015348229557275772, 1.4756145477294922, 0.015285491943359375, 0.0010789170628413558, 0.6433061361312866, 1.0542027950286865, 0.04665156826376915, 0.3687209188938141, 0.31800156831741333, 0.23083575069904327, 0.1742555946111679, 0, 0.14921215176582336, 0.23223736882209778, 0, 0.22727835178375244, 0, 0, 0.09276270866394043, 0.5781478881835938, 0.24056950211524963, 0.10346803814172745, 1.0351626873016357, 0.5553533434867859, 0.044809453189373016, 0, 0, 0, 0.358695387840271, 0.29990673065185547, 0, 0, 0, 0.10074195265769958, 0.19350290298461914, 0.17824490368366241, 0.03202737122774124, 0.0376395583152771, 0.03683996945619583, 0, 0, 1.3127859830856323, 0, 0, 0.1427265852689743, 0.1927458792924881, 0, 0, 0, 0.24242720007896423, 0, 0, …] 'found'

console.log(Array.from(this.descriptor_ids[i]),'db')

(1024) [NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, …] 'db'

so its clearly serializized result that is stored in your db that is wrong, not result.face[0].embedding itself

[shurshilov]

from YOUR DEMO:

const found=result.face[0].embedding
console.log(found,'found')

(1024) [0.2984731197357178, 0.2595631778240204, 0.3963339328765869, 0, 0, 0, 0.23985564708709717, 0.7768914103507996, 1.177507758140564, 0.7845110893249512, 0, 0.02752731181681156, 0, 0.09198596328496933, 0.4361388087272644, 0, 0.18945324420928955, 0.0012000531423836946, 0.005083911120891571, 0, 0.36032697558403015, 0.035741083323955536, 0.07470899820327759, 0, 0, 0, 1.1794145107269287, 0.9509024024009705, 0.1838228404521942, 0.1625743806362152, 0.012997782789170742, 0.15879793465137482, 0, 1.0624027252197266, 1.2277904748916626, 0, 0.002714419737458229, 0.014698048122227192, 0, 0, 0.11073991656303406, 0, 0.21221040189266205, 0, 0, 0, 0, 0.4801158308982849, 0.015348229557275772, 1.4756145477294922, 0.015285491943359375, 0.0010789170628413558, 0.6433061361312866, 1.0542027950286865, 0.04665156826376915, 0.3687209188938141, 0.31800156831741333, 0.23083575069904327, 0.1742555946111679, 0, 0.14921215176582336, 0.23223736882209778, 0, 0.22727835178375244, 0, 0, 0.09276270866394043, 0.5781478881835938, 0.24056950211524963, 0.10346803814172745, 1.0351626873016357, 0.5553533434867859, 0.044809453189373016, 0, 0, 0, 0.358695387840271, 0.29990673065185547, 0, 0, 0, 0.10074195265769958, 0.19350290298461914, 0.17824490368366241, 0.03202737122774124, 0.0376395583152771, 0.03683996945619583, 0, 0, 1.3127859830856323, 0, 0, 0.1427265852689743, 0.1927458792924881, 0, 0, 0, 0.24242720007896423, 0, 0, …] 'found'

console.log(Array.from(this.descriptor_ids[i]),'db')

(1024) [NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, NaN, …] 'db'

so its clearly serializized result that is stored in your db that is wrong, not result.face[0].embedding itself

How its possiible//my browser print this

So i think problem in some versions of browser....hmm I updated browser to Chrome 105.0.5195.54 (Official build), (64 bit) error still I also try Microsoft Edge Version104.0.1293.70 error still

[vladmandic]

i just tried with chrome 104/105/106/107 and same results - result.face[0].embedding is a valid array, serialization is not. same with firefox

disable browser cache and make sure correct models are loaded.

[shurshilov]

i just tried with chrome 104/105/106/107 and same results - result.face[0].embedding is a valid array, serialization is not. same with firefox

disable browser cache and make sure correct models are loaded.

Very strange but we use one page... so problem only in device can be...

[vladmandic]

regarding serialization, avoid looping and doing things such as charCodeAt
very simple approach:

  // serialize
  const blob = new Blob([Float32Array.from(result.face[0].embedding)]); // store fixed-size 4k blob to database (4 bytes per float entry) and if you need to store text, you can use blob.text()
  // deserialize
  const embedding = new Float32Array(await blob.arrayBuffer());
   // verify that they are the same -> returns 100%
  const similarity = human.similarity(result.face[0].embedding, embedding);

[shurshilov]

regarding serialization, avoid looping and doing things such as charCodeAt very simple approach:

  // serialize
  const blob = new Blob([Float32Array.from(result.face[0].embedding)]); // store fixed-size 4k blob to database (4 bytes per float entry) and if you need to store text, you can use blob.text()
  // deserialize
  const embedding = new Float32Array(await blob.arrayBuffer());
   // verify that they are the same -> returns 100%
  const similarity = human.similarity(result.face[0].embedding, embedding);

I'm telling you in my result device.face[0].embedding returns an array with Nan. NO serialization, just a handle call. The fact that the arrival of emptiness from the database is just the same bug, this empty array has been preserved. If we visited the same page. With the same browsers and you get found with numbers, and I have Nan, so the problem is in the device... or something else that I do not know) that's why I sent you a screenshot where found with Nan and found is just console.log(result.face[0].embedding, 'found')

[shurshilov]

Yes my adnroid 11 chrome work good on 2.9.1. So after 2.6.3 break some device capability.. My device error AMD Ryzen 7 5800H with Radeon Graphics 3.20 GHz Windows 10 Pro 64 USB2.0 HD UVC Webcam - camera

2videocards because notebook AMD Radeon graphics Nvidia gforce RTX 3060 Laptop GPU

[vladmandic]

can you try https://vladmandic.github.io/human/demo/?backend=webgl select menu -> display -> results tree and start video after few frames, pause it and expand results -> face -> 0 -> embedding. what do you see?

also try for https://vladmandic.github.io/human/demo/?backend=wasm

and last one, try https://vladmandic.github.io/human/demo/facematch/ once it finishes (it takes some time to analyze all test photos), click on any face - and in browser console it will print results.face[0] for selected face - what do you see there?

[shurshilov]

can you try https://vladmandic.github.io/human/demo/?backend=webgl select menu -> display -> results tree and start video after few frames, pause it and expand results -> face -> 0 -> embedding. what do you see?

also try for https://vladmandic.github.io/human/demo/?backend=wasm

and last one, try https://vladmandic.github.io/human/demo/facematch/ once it finishes (it takes some time to analyze all test photos), click on any face - and in browser console it will print results.face[0] for selected face - what do you see there? webgl work wasm work facematch error

MAy be compare chrome://gpu/ ?

Graphics Feature Status Canvas: Hardware accelerated Canvas out-of-process rasterization: Enabled Direct Rendering Display Compositor: Disabled Compositing: Hardware accelerated Multiple Raster Threads: Enabled OpenGL: Enabled Rasterization: Hardware accelerated Raw Draw: Disabled Video Decode: Hardware accelerated Video Encode: Hardware accelerated Vulkan: Disabled WebGL: Hardware accelerated WebGL2: Hardware accelerated WebGPU: Hardware accelerated Driver Bug Workarounds clear_uniforms_before_first_program_use decode_encode_srgb_for_generatemipmap disable_delayed_copy_nv12 disable_direct_composition_sw_video_overlays disable_nv12_dynamic_textures disable_vp_scaling disable_vp_super_resolution disallow_vp9_resilient_dxva_decoding enable_webgl_timer_query_extensions exit_on_context_lost limit_d3d11_video_decoder_to_11_0 use_empty_video_hdr_metadata disabled_extension_GL_KHR_blend_equation_advanced disabled_extension_GL_KHR_blend_equation_advanced_coherent Problems Detected Some drivers are unable to reset the D3D device in the GPU process sandbox Applied Workarounds: exit_on_context_lost

[shurshilov]

I found ONLY https://vladmandic.github.io/human/demo/?backend=humangl not working

[shurshilov]

when I changed settings browser to

from

That is, if I disable hardware acceleration in the settings, then it starts working. And if I turn it on it doesn't work

[vladmandic]

this is really weird. no differences in chrome://gpu, only thing that is obvious is different system locale, but that should not matter at all

just to confirm:

• pass: https://vladmandic.github.io/human/demo/?backend=webgl
• pass: https://vladmandic.github.io/human/demo/?backend=wasm
• fail: https://vladmandic.github.io/human/demo/?backend=humangl

since i cannot reproduce, i wrote a quick test with several possibilities
can you run https://vladmandic.github.io/human/test/test-embedding.html
and copy and paste entire screen output here

[shurshilov]

this is really weird. no differences in chrome://gpu, only thing that is obvious is different system locale, but that should not matter at all

just to confirm:

• pass: https://vladmandic.github.io/human/demo/?backend=webgl
• pass: https://vladmandic.github.io/human/demo/?backend=wasm
• fail: https://vladmandic.github.io/human/demo/?backend=humangl

since i cannot reproduce, i wrote a quick test with several possibilities can you run https://vladmandic.github.io/human/test/test-embedding.html and copy and paste entire screen output here

As I wrote in the last message, I seem to have found an error. If I turn off hardware acceleration, then humangl starts working for me. If I turn it on, it stops working. Here is the output from your test

human 2.9.4 {config: {backend: humangl, modelBasePath: https: //vladmandic.github.io/human-models/models/, cacheModels: true, validateModels: true, wasmPath: https: //cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-wasm@3.20.0/dist/, wasmPlatformFetch: false, debug: true, async: false, warmup: face, cacheSensitivity: 0, skipAllowed: false, deallocate: false, softwareKernels: false, filter: {enabled: true, equalization: false, width: 0, height: 0, flip: false, return: true, brightness: 0, contrast: 0, sharpness: 0, blur: 0, saturation: 0, hue: 0, negative: false, sepia: false, vintage: false, kodachrome: false, technicolor: false, polaroid: false, pixelate: 0}, gesture: {enabled: true}, face: {enabled: true, detector: {modelPath: blazeface.json, rotation: true, maxDetected: 1, skipFrames: 99, skipTime: 2500, minConfidence: 0.2, iouThreshold: 0.1, mask: false, return: false}, mesh: {enabled: true, modelPath: facemesh.json, keepInvalid: false}, attention: {enabled: false, modelPath: facemesh-attention.json}, iris: {enabled: false, modelPath: iris.json}, emotion: {enabled: false, minConfidence: 0.1, skipFrames: 99, skipTime: 1500, modelPath: emotion.json}, description: {enabled: true, modelPath: faceres.json, skipFrames: 99, skipTime: 3000, minConfidence: 0.1}, antispoof: {enabled: false, skipFrames: 99, skipTime: 4000, modelPath: antispoof.json}, liveness: {enabled: false, skipFrames: 99, skipTime: 4000, modelPath: liveness.json}}, body: {enabled: false, modelPath: movenet-lightning.json, maxDetected: -1, minConfidence: 0.3, skipFrames: 1, skipTime: 200}, hand: {enabled: false, rotation: true, skipFrames: 99, skipTime: 1000, minConfidence: 0.5, iouThreshold: 0.2, maxDetected: -1, landmarks: true, detector: {modelPath: handtrack.json}, skeleton: {modelPath: handlandmark-full.json}}, object: {enabled: false, modelPath: mb3-centernet.json, minConfidence: 0.2, iouThreshold: 0.4, maxDetected: 10, skipFrames: 99, skipTime: 2000}, segmentation: {enabled: false, modelPath: selfie.json, blur: 8}, gestures: {enabled: false}}} wasm {backend: wasm, env: {browser: true, node: false, worker: false, platform: Windows NT 10.0; Win64; x64, agent: Mozilla/5.0 AppleWebKit/537.36 Chrome/105.0.0.0 Safari/537.36, backends: [cpu, webgl, wasm], initial: false, tfjs: {version: 3.20.0}, offscreen: true, perfadd: false, tensorflow: {}, wasm: {supported: true, backend: true, simd: true, multithread: false}, webgl: {supported: true, backend: true}, webgpu: {supported: false, backend: false}, cpu: {flags: []}, kernels: 110}} wasm {backend: wasm, warmup: none} wasm {backend: wasm, warmup: face, gender: female, genderScore: 0.93, age: 23.5, embedding: [0, 0, 0, 0.016783053055405617, 0.39307835698127747, 0.02448403649032116, 0.3376886546611786, 0, 0, 0]} wasm {backend: wasm, warmup: body, gender: female, genderScore: 0.97, age: 29.6, embedding: [0, 0, 0.001799826743081212, 0.4305435121059418, 0, 0.09787886589765549, 0.042208097875118256, 0, 0, 0]} webgl {backend: webgl, env: {browser: true, node: false, worker: false, platform: Windows NT 10.0; Win64; x64, agent: Mozilla/5.0 AppleWebKit/537.36 Chrome/105.0.0.0 Safari/537.36, backends: [cpu, webgl, wasm], initial: false, filter: true, tfjs: {version: 3.20.0}, offscreen: true, perfadd: false, tensorflow: {}, wasm: {supported: true, backend: true, simd: true, multithread: false}, webgl: {supported: true, backend: true, version: WebGL 2.0 (OpenGL ES 3.0 Chromium), renderer: WebKit WebGL}, webgpu: {supported: false, backend: false}, cpu: {flags: []}, kernels: 166}} webgl {backend: webgl, warmup: none} webgl {backend: webgl, warmup: face, gender: male, genderScore: 0.13, age: 29.3, embedding: [0, 0.1438342183828354, 0, 0, 0, 0, 0, 0.05724700540304184, 0.020496606826782227, 0]} webgl {backend: webgl, warmup: body, gender: male, genderScore: 0.26, age: 29.3, embedding: [0, 0, 0, 0, 0, 0, 0, 0.08994250744581223, 0, 0]} humangl {backend: humangl, env: {browser: true, node: false, worker: false, platform: Windows NT 10.0; Win64; x64, agent: Mozilla/5.0 AppleWebKit/537.36 Chrome/105.0.0.0 Safari/537.36, backends: [cpu, webgl, wasm, humangl], initial: false, filter: true, tfjs: {version: 3.20.0}, offscreen: true, perfadd: false, tensorflow: {}, wasm: {supported: true, backend: true, simd: true, multithread: false}, webgl: {supported: true, backend: true, version: WebGL 2.0 (OpenGL ES 3.0 Chromium), renderer: WebKit WebGL}, webgpu: {supported: false, backend: false}, cpu: {flags: []}, kernels: 166}} humangl {backend: humangl, warmup: none} humangl {backend: humangl, warmup: face, gender: unknown, embedding: [null, null, null, null, null, null, null, null, null, null]} humangl {backend: humangl, warmup: body, gender: unknown, embedding: [null, null, null, null, null, null, null, null, null, null]} webgpu {desired: webgpu, detected: humangl} test1 {backend: humangl, env: {browser: true, node: false, worker: false, platform: Windows NT 10.0; Win64; x64, agent: Mozilla/5.0 AppleWebKit/537.36 Chrome/105.0.0.0 Safari/537.36, backends: [cpu, webgl, wasm, humangl], initial: false, filter: true, tfjs: {version: 3.20.0}, offscreen: true, perfadd: false, tensorflow: {}, wasm: {supported: true, backend: true, simd: true, multithread: false}, webgl: {supported: true, backend: true, version: WebGL 2.0 (OpenGL ES 3.0 Chromium), renderer: WebKit WebGL}, webgpu: {supported: false, backend: false}, cpu: {flags: []}, kernels: 166}} test1 {backend: humangl, warmup: none} test1 {backend: humangl, warmup: face, gender: unknown, embedding: [null, null, null, null, null, null, null, null, null, null]} test1 {backend: humangl, warmup: body, gender: unknown, embedding: [null, null, null, null, null, null, null, null, null, null]} test2 {backend: humangl, env: {browser: true, node: false, worker: false, platform: Windows NT 10.0; Win64; x64, agent: Mozilla/5.0 AppleWebKit/537.36 Chrome/105.0.0.0 Safari/537.36, backends: [cpu, webgl, wasm, humangl], initial: false, filter: true, tfjs: {version: 3.20.0}, offscreen: true, perfadd: false, tensorflow: {}, wasm: {supported: true, backend: true, simd: true, multithread: false}, webgl: {supported: true, backend: true, version: WebGL 2.0 (OpenGL ES 3.0 Chromium), renderer: WebKit WebGL}, webgpu: {supported: false, backend: false}, cpu: {flags: []}, kernels: 166}} test2 {backend: humangl, warmup: none} test2 {backend: humangl, warmup: face, gender: unknown, embedding: [null, null, null, null, null, null, null, null, null, null]} test2 {backend: humangl, warmup: body, gender: unknown, embedding: [null, null, null, null, null, null, null, null, null, null]} test3 {backend: humangl, env: {browser: true, node: false, worker: false, platform: Windows NT 10.0; Win64; x64, agent: Mozilla/5.0 AppleWebKit/537.36 Chrome/105.0.0.0 Safari/537.36, backends: [cpu, webgl, wasm, humangl], initial: false, filter: true, tfjs: {version: 3.20.0}, offscreen: true, perfadd: false, tensorflow: {}, wasm: {supported: true, backend: true, simd: true, multithread: false}, webgl: {supported: true, backend: true, version: WebGL 2.0 (OpenGL ES 3.0 Chromium), renderer: WebKit WebGL}, webgpu: {supported: false, backend: false}, cpu: {flags: []}, kernels: 166}} test3 {backend: humangl, warmup: none} test3 {backend: humangl, warmup: face, gender: unknown, embedding: [null, null, null, null, null, null, null, null, null, null]} test3 {backend: humangl, warmup: body, gender: unknown, embedding: [null, null, null, null, null, null, null, null, null, null]} test4 {backend: humangl, env: {browser: true, node: false, worker: false, platform: Windows NT 10.0; Win64; x64, agent: Mozilla/5.0 AppleWebKit/537.36 Chrome/105.0.0.0 Safari/537.36, backends: [cpu, webgl, wasm, humangl], initial: false, filter: true, tfjs: {version: 3.20.0}, offscreen: true, perfadd: false, tensorflow: {}, wasm: {supported: true, backend: true, simd: true, multithread: false}, webgl: {supported: true, backend: true, version: WebGL 2.0 (OpenGL ES 3.0 Chromium), renderer: WebKit WebGL}, webgpu: {supported: false, backend: false}, cpu: {flags: []}, kernels: 166}} test4 {backend: humangl, warmup: none} test4 {backend: humangl, warmup: face, gender: unknown, embedding: [null, null, null, null, null, null, null, null, null, null]} test4 {backend: humangl, warmup: body, gender: unknown, embedding: [null, null, null, null, null, null, null, null, null, null]}

[vladmandic]

humangl compared to webgl

• forces webgl v2 (which your system does support)
• implements some additional webgl error handling (which is not an issue since any errors would be logged on console)
• sets couple of performance tuning params

so its really strange that it doesn't work while webgl does work. since i cannot reproduce, i'm disabling performance tunes one-by-one, can you try https://vladmandic.github.io/human/test/test-embedding.html again?
no need for copy&paste, just say which test is pass or fail - there will be default webgl, humangl and 4x test

[shurshilov]

humangl compared to webgl

• forces webgl v2 (which your system does support)
• implements some additional webgl error handling (which is not an issue since any errors would be logged on console)
• sets couple of performance tuning params

so its really strange that it doesn't work while webgl does work. since i cannot reproduce, i'm disabling performance tunes one-by-one, can you try https://vladmandic.github.io/human/test/test-embedding.html again? no need for copy&paste, just say which test is pass or fail - there will be default webgl, humangl and 4x test

I dont know how check test good or not, then send you screen)) but all test give null

[vladmandic]

if it says null inside any embedding results, its a fail - in your case webgl is pass and all other tests are fail. if you look at browser console, is there anything printed between those two lines

Human: available backends: (4) ['cpu', 'webgl', 'wasm', 'humangl']
Human: setting backend: humangl

i've added extra diagnostic output to library itself, if you run again, can you capture console output like this (if its missing, wait 5min for git pages to refresh)?

[shurshilov]

[vladmandic]

i've changed some backend init code, can you rerun and screenshot entire console output? should look like:

btw, sorry for going back-and-forth, its just i cannot reproduce and this is a really weird problem.

[shurshilov]

Not problem I sold module with your libs more then 15 times) and my customers always have question for example it also not work with android 12

[vladmandic]

this is old screenshot, see how it should look - i guess gitpages was not updated (it is now). (btw, its best to open browser with inspector open and then paste url - chrome prints differently if inspector is already open when loading page or when opening inspector later)

you say it doesn't work on android 12? i just checked on s21 and s10, both with android 12 and it works ok here.

[shurshilov]

About Android 12 it was customer fail( sorry.. so page not updated?

[vladmandic]

This is the output I needed. But everything is exactly as expected - there is nothing out of ordinary. I need to think about it.

In the meantime, I suggest thar for your app you set config.backend = 'webgl' so you're not blocked.

It really makes very little difference in real-life, but I do want to figure this out.

[vladmandic]

one last test when you have the chance: https://vladmandic.github.io/human/test/test-embedding.html

i've decided to backport two remaining performance features from humangl to webgl and use webgl as default moving forward.
main reason for humangl was that i used custom version of tfjs in the past as official version of tfjs was quite behind in implementing some required fixes. now that its up-to-date, there is no real reason to use custom backend. its still present, but default is back to webgl.

if there are no issues, changes will be published as part of new 2.10 release early next week.

[shurshilov]

Hi)

[vladmandic]

thanks. i did not expect this, but at least this narrows it down to only two possibilities. unfortunately, it impacts how models are compiled during load & warmup.

ok, re-run same test - https://vladmandic.github.io/human/test/test-embedding.html

but there are two checkboxes - run test with none selected, just first, just second and both.
and you need to reload page between tests.

only thing that matters is last line of output of each test - please copy&paste that here.

[vladmandic]

any updates?

[shurshilov]

any updates?

[vladmandic]

silly typo. fixed. sorry.

[shurshilov]

[vladmandic]

ok, so now we i have a confirmation of a root cause, i'll work with tfjs team to figure it out

i don't want to globally disable this feature as it has significant performance advantages

fyi, this feature enables usage of gl uniforms instead of textures in some places in glsl code so it speeds up model compilation as multiple models can be warmed-up/compiled in parallel so time to first result is about 2x faster

so you can disable it for your app for now:

• in human versions 2.7 - 2.9 you can simply set config.backend = 'webgl'
  as optimizations are not enabled for default webgl backend, only for my port which is humangl
• in human versions 2.10 (will be released this week), you can explicitly
  disable this optimization with config.flags.WEBGL_USE_SHAPES_UNIFORMS = false

[shurshilov]

Good!Thank you

[vladmandic]

@shurshilov i'm going to close this issue here - lets continue conversation in https://github.com/tensorflow/tfjs/issues/6822
(please provide info tfjs devs need when you can)