Head Tracking

[GoogleCodeExporter]

I was just wondering if your software will support head tracking via the Kinect 
RGB camera.  I've read that head tracking is limited with the depth IR camera. 

Original issue reported on code.google.com by kofian...@gmail.com on 4 Jul 2011 at 6:06

[GoogleCodeExporter]

its possible. I hadn't planed to child that yet, but I'm not ruling it out 
either.

If you could write yo a list of the exact goals of head taking, it would help 
me visualize the production of it and how useful it would be, compared top the 
other features that are in the works

Thanks! 

Original comment by gareth.p...@gmail.com on 4 Jul 2011 at 8:09

• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

I mainly imagine it being useful in most games related to operating a vehicle 
and FPS games.  Consider that tools like TrackIR5 has been implemented in 
racing games. Games like GTAIV and Mafia 2 or even the battlefield series have 
heavy use of vehicles. 

I saw a Demo that used head tracking to look down sites (tilt of the head) and 
look around corners.  I would mainly like to use it to look around in the 
gameworld with out using my hand for mouse movement when not using a gun. Head 
Tracking could be activated by a gesture.  I imagine that i would integrate a 
wiimote for extra realism (using a trigger to shoot).  I could use any wiimote 
button to activate head tracking mode. Side note: If i could still hand track 
with a wiimote in hand it would be amazing (the wiimote ir on the PC is crap 
outside of Dolphin) 

Original comment by kofian...@gmail.com on 4 Jul 2011 at 10:47

[GoogleCodeExporter]

Things like that still need to be supported by the game you are using it on.

I know that Doom2(skulltag) has a "look left" / "look right" ability... but 
nothing I know of has functionality like that Kinect racing game coming out 
with the ability to look around the cockpit.

What PC games do you know of that have a control for looking around that is 
separate from your directional controls?

Original comment by gareth.p...@gmail.com on 5 Jul 2011 at 5:03

[GoogleCodeExporter]

I'm aware of the limitations with in game controls.  What I'm imagining is the 
ability to look around when I'm not shooting.  Imagine the head taking over for 
the mouse when not shooting or when in a vehicle. TrackIR5 is used in games 
like grid or other racing games (or games involving driving ie GTA IV) to look 
around when shooting is not required. When using your arm to aim, you are 
controlling the mouse to look around by default.

Aiming isn't always required and this would allow you to rest your arm when 
you're not in aim mode when it is only necessary to observe your environment. A 
command could be used to holster or take out your weapon which would intern 
switch from your head to your hand to look around\Aim.

Original comment by kofian...@gmail.com on 5 Jul 2011 at 7:31

[GoogleCodeExporter]

I see

Well it's possible, will take some tweaking to get done though. :)

any details you can think of would help. As far as how to react to movement, 
other than the basic pointing.

Does it keep going right if my head is twisted right? how to handle it properly 
so you can still SEE the screen while using this head turning mode?

details like that

Original comment by gareth.p...@gmail.com on 5 Jul 2011 at 11:03

[GoogleCodeExporter]

I imagine a bounding box could be implemented relative to the rotation of
the head.  I saw a video that allowed the character to do a complete 360
when twisting the head twice in either direction.  Sort of like a double
look to the left or right.  The bounding box could 45 degrees or less.
Imagine how the Kinect reacts when detecting the turn of the body or the
position of the arm relative to the resolution of the monitor.

The RGB camera should be able to detect the nose eyes and mouth.  I don't
know how much the Kinect SDK gives you to work with when dealing with the
RGB camera.  I did read that the data from the RGB camera can be combined
with the depth sensor data.  Maybe that can aid in accurately detecting the
original position of the entire head and its absolute center.  I know that
the depth camera can not detect the details like the nose or eyes (i would
think that it could being that it can detect individual fingers).

Take a look at these videos for examples.

The first video i just found gives good details explaining how TrackIR works
with *ArmA2*
http://www.youtube.com/watch?v=9wXx3vMy_AQ

This is* TrackIR 5 *with* GTA IV, HAWX, GRID*
http://www.youtube.com/watch?v=7Hu8MzhW9p8 (displays walking around also)

http://www.youtube.com/watch?v=UHhd1FD-NYQ&feature=related

http://www.youtube.com/watch?v=VOCxiFpR93E&feature=related

------------

Some other guys pulled off head tracking with the Kinect but this is pre
SDK.
http://www.youtube.com/watch?v=CddvvlP4UNM

This did lots with the Kinect to show off it's gaming ability when using
FAAST but implemented the wii for head tracking.  Does the Kinect SDK
provide full access to all features?  I would imagine that to be the case.

http://www.youtube.com/watch?v=ZWGhN99BJBE

This is an application called freetrack that uses webcams to track the face
(RGB Camera?)
http://www.youtube.com/watch?v=MEnD0jitcII

On Tue, Jul 5, 2011 at 7:03 PM, <
kinect-activity-reaction-interface@googlecode.com> wrote:

Original comment by kofian...@gmail.com on 6 Jul 2011 at 12:06