Controller state and hand tracking?

[JeroMiya]

It would be nice if we could implement both controller state and hand tracking in the OSVR plugin.

Questions to answer (feel free to answer them in a reply below, I'll update the issue):

1. Do we add a new tracked device for each hand, or are the tracked devices part of the controller state?
2. Can we add controller state without a tracked hand? Example: OSVR supports an Xbox controller via the VRPN driver. In this case we don't have a tracked hand but we have a controller.
3. Can we have a tracked hand without controller state? Example: Leap motion or Nod or other device that tracks hands but doesn't have buttons (unless you mapped them to gestures).
4. If we return a new tracked device for each hand, how do we tell it we're the "left hand" or the "right hand"?

[Minostaurus0]

As far I could see, there is no support for controllers in the code for now. I managed to add this support. For now both controllers are detected by SteamVR and they are sending some dummy data to SteamVR. Next I will try to add the interface to the OSVR-Tree and then I will make a Pull request.

1. Each hand is an own tracked device.
2. If I understand you right: yes.
3. We don't need to send the button states, so yes.
4. I don't know at the moment.

[DjDyll]

1. Each hand is its own tracked device.
2. Yes.
3. yes
4. is it possible to split the tracking in two halves, or maybe track the thumb.

[rpavlik]

Fwiw, just did an implementation of the other side of this with the OSVR-Vive plugin, which acts like a partial implementation of steamvr (the driver loading side). I've put a few notes below in case they're helpful, if anyone gets stuck I probably now have something to offer here, at least as far as observed behavior of what the first party Vive plugin is sending to steam vr.

Everything is a tracked device first and foremost. Some devices are also other types (inherit from other "component" bases, like display, or controller), which are accessible via a get component call that is vaguely like query interface. Most of the reporting is done through calling methods of a "server driver host", and so I don't remember offhand what exactly the controller component adds to the mix besides identifying a symbolic role: I do know that the Vive HMD itself has a button on it, that I'm pretty sure it just reports normally like other buttons, but I know it does not report a controller component.

Buttons are in a bit mask, iirc, and analogs are in pairs called axes, with some semantic labeling (from enums) possible but limited (header comment notes that the analog trigger is in the x channel of its pair of axes, for instance)

Tracked devices can also have some properties, some of which are relatively required, some are relatively obscure, and knowing the difference is anyone's guess: the only really important one is ID aka serial number since it's used as a system wide unique ID, at least within (not sure if between or not) a session, to identify a data source (a " tracked device ") - I'd suggest deriving something from the fully resolved original source of an alias. (The catch is that this is far more granular: it's one data stream from one device - but as long as it's deterministic it should be fine)

Doing this right will involve using some more of the internal APIs (mainly a few pieces in OSVR common) of OSVR to introspect/traverse the path tree.

On Fri, Mar 18, 2016, 12:53 PM DjDyll notifications@github.com wrote:

1. Each hand is its own tracked device.
2. Yes.
3. yes
4. is it possible to split the tracking in two halves, or maybe track the thumb.

— You are receiving this because you are subscribed to this thread.

Reply to this email directly or view it on GitHub https://github.com/OSVR/SteamVR-OSVR/issues/30#issuecomment-198469855

Ryan A. Pavlik, Ph.D. Senior Software Engineer Sensics, Inc. www.sensics.com

[rpavlik]

Re left and right hand, I think they take controllers as ambidextrous and assign hands up at a middle ware level: there appears to be nothing down at the driver level about that. OSVR would have a plugin watching the head and wands and eventually adding left and right aliases to the tree; in steamvr while the left and right hand controller enumeration is in the driver header it looks like an accident, because it's not actually used anywhere, or at least anywhere publicly.

[Conzar]

Additional Notes:

• The start of an implementation: Controller Branch
• Reference for Razer Hydra: git repo

I will attempt at updating the controller branch soon. My motivation is getting this working for the NoloVR controller tracking.