vladmandic / human

Human: AI-powered 3D Face Detection & Rotation Tracking, Face Description & Recognition, Body Pose Tracking, 3D Hand & Finger Tracking, Iris Analysis, Age & Gender & Emotion Prediction, Gaze Tracking, Gesture Recognition
https://vladmandic.github.io/human/demo/index.html
MIT License
2.34k stars 323 forks source link

False Positives and False Negatives in Hand Detection & Linking faces with hands. #497

Closed ceyxasm closed 5 days ago

ceyxasm commented 1 week ago

Issue Description Model giving a high number of false positives (minConfidence: 0.75). Also has a fair share of false negatives.

Human config:

const myConfig = {
  modelBasePath: 'file://models/',
  debug: true,
  filter: { enabled: false, equalization: false, flip: false },
  face: {
    enabled: true,
    detector: {
      maxDetected: 100,
      rotation: false,
      skipFrames: 30,
      minConfidence: 0.5
    },
    mesh: {enabled: false},
    attention: {enabled: false},
    iris: {enabled: false},
    description: {enabled: false},
    emotion: {
      enabled: false,
      skipFrames: 30,
    }
  },
  person: {
    enabled: true
  },
  body: { enabled: false },
  hand: { enabled: true, maxDetected: 100, minConfidence: 0.75 },
  gesture: { enabled: false },
  object: { enabled: false },
  segmentation: { enabled: false },
};

Hands are supposed to be in green boxes image image image


My second concern is how can I link a particular hand with a particular face (/human). IDs in result.hand are independent of IDs in result.face

vladmandic commented 5 days ago

human's hand module was never intended for this. you can try using alternative hand detection model (it supports handdetect as alternative in addition to handtrack which is default) which is more reliable, but much slower.

and try with less extreme inputs - use large resolution square images where each person is separate. having really skewed aspect ratios and people overlapping by more than 50% is drastically reducing any chance of sane results.

and then you can look at per-person unified results using human.person getter that recombines all different results into per-person view. but for that to work, you also need to have body detection which you've turned off.