Face IDs shift when a face disappears

[NyaomiDEV]

Issue Description

In tracking multiple faces, when one person leaves the area, all face IDs are shifted by one position down. This makes relying on ID dodgy at times.

Steps to Reproduce

1. Setup Human (the live "full" demo is also okay - watch how the boxes change color)
2. Walk away from the webcam when another person is still in frame
3. Observe issue

Expected Behavior

The in-frame faces should be always identifiable with the same ID, or something, until they leave the frame.

Environment

• Human library version? 3.2.1
• Built-in demo or custom code? Custom code (reproduced on "Full" demo too)
• Type of module used (e.g. js, esm, esm-nobundle)? js
• TensorFlow/JS version (if not using bundled module)? 4.17.0
• Browser or NodeJS and version (e.g. NodeJS 14.15 or Chrome 89)? NodeJS 20 / Firefox 122
• OS and Hardware platform (e.g. Windows 10, Ubuntu Linux on x64, Android 10)? Arch Linux on x64
• Packager (if any) (e.g, webpack, rollup, parcel, esbuild, etc.)? No packager
• Framework (if any) (e.g. React, NextJS, etc.)? No framework

Additional

const humanConfig = {
    backend: "tensorflow",
    modelBasePath: `file://${modelUrl}`,
    debug: false,
    async: true,
    face: {
        enabled: true,
        detector: {
            maxDetected: 10,
            minConfidence: 0.3
        },
        gear: {
            enabled: false
        },
        mesh: {
            enabled: true
        },
        antispoof: {
            enabled: false
        },
        attention: {
            enabled: false
        },
        iris: {
            enabled: false
        },
        description: {
            enabled: false
        },
        emotion: {
            enabled: true
        },
        liveness: {
            enabled: false
        }
    },
    body: {
        enabled: false
    },
    hand: {
        enabled: false
    },
    object: {
        enabled: false
    },
    gesture: {
        enabled: false
    },
    segmentation: {
        enabled: false
    }
};

async function detect(image: Buffer) {
    if (human.state !== "idle")
        return;

    const tensor = human.tf.node.decodeJpeg(image);
    await human.detect(tensor);
    const res = human.next();

    console.log(res.face.map(x => x.id), res.face.length);

[vladmandic]

The in-frame faces should be always identifiable with the same ID, or something, until they leave the frame.

I have experimented with that in details, but the problem is much larger than you'd expect. for example, simple scenario with two people: what if one person walks in front of another so they temporarily overlap and then they separate again? which one is which?

i've tried some simple predictive analysis based on trajectory between different frames, but its unreliable. and anything more would require significant calculations that would slow down the app drastically.

and you have all the information available - so you can do that outside of the library - this is not something that i plan to add as a feature inside the library itself.