AIRLegend / aitrack

6DoF Head tracking software
MIT License
1.03k stars 102 forks source link

version 0.6.6 alpha randomly losing tracking stuck in face down position #161

Closed searching46dof closed 1 year ago

searching46dof commented 1 year ago

Describe the bug A clear and concise description of what the bug is.

version 0.6.6alpha seems to be randomly losing tracking and returns coordinates in face down position for 3 to 5secs. there does not seem to be correlation to the trigger (actual face may be looking up,down,left,right or forward position when it occurs).

0.6.6alpha returns more of a pitch range (+/-45 deg max) than 0.6.5alpha (+/-30 deg max) so I thought it may be the interaction with opentrack's filters but it seems to occur with all of opentrack filters.

when it occurs, raw pitch jitters between +30 and +35 deg(=down) while raw yaw jitters around 0 deg(center) and raw roll jitters around 90 (upright center) deg. moving my head around when this occurs doesn't seem to affect the other coordinates (yaw,roll,x,y,z) switching back to version 0.6.5 does not have this issue.

roll in version 0.6.6 seems to be limited in one direction <=90deg (90=upright center). the range of roll should be restricted to 0 - 180deg. realistically roll does not need to be bounded since it loses track at approx +/-45deg from center but this may depend on the score_threshold.

To Reproduce Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

It appears to be readily reproducible when flying in DCS and then stuck looking at the floor of the cockpit. There is no one else in camera view when this occurs and my face appears to be the only one detected (switched screens when it occurred)

When testing AITrack standalone, version 0.6.6 seems to produce some face detection false alarms from random background objects (e.g. smaill pile of clothes) when the gain/exposure is a little too high. Maybe cv::FaceDetectorYN::create should have configurable score_threshold and nms_threshold in prefs.ini ?

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

Additional context Add any other context about the problem here.

AIRLegend commented 1 year ago

I've not experienced this issue.

Have you checked if, in the preview window, the blue detection box disappears?

The non-max-suppression (nms) parameter will filter bounding boxes that are "pointing" to the same face (i.e. overlapping). It'd be better off leaving it hardcoded and not exposing it to the user (we should avoid flooding the average user with settings). It could be changed though. I set it as what I thought it was reasonable, but it could probably be decreased even more.

Increasing score_threshold could avoid those false positives. But I think your "center weighted face selection" change in #162 should fix it. Does it still happen using that new behaviour?

searching46dof commented 1 year ago

So far I was getting this behavior in DCS so I couldn't see the preview window.

I tried duplicating the symptoms with just AItrack and it may be related to cv::FaceDetectorYN losing tracking. e.g. 1) look left or right and the preview window face tracking corresponds to the movement. opentrack squid also follows correctly. 2) while still facing left or right, cover the bottom half of your face (e.g. with a magazine or keyboard) until it loses tracking. the opentrack squid then looks down (also indicated by the raw udp data)

searching46dof commented 1 year ago

The loss of tracking seems to be related to insufficient margins for the bounding box around the face.

The symptom of looking down when losing tracking seems to be specific to version 0.6.6. So far I could not reproduce it with version 0.7.0.

searching46dof commented 1 year ago

I think this issue was resolved in 0.7.0. I can't reproduce it in 0.7.0. or 0.7.1