Unity Plugin for accessing HoloLens2 Research Mode sensors and video camera, and streaming them to desktop. It builds upon the official HoloLens2ForCV and HoloLensForCV repos.
Currently, only Depth AHAT and video camera streams are enabled, but an extension to other RM sensors should be straight forward and will probably be added in the future. The image, as well as rig2world
transforms for AHAT and and pv2world
, fx
and fy
for video camera are transmitted for each frame.
HL2RmStreamUnityPlugin/Dependencies
.Release, ARM64
.Assets/Plugins/WSAPlayer/ARM64
.HL2RmStreamUnityPlugin.dll
from HL2RmStreamUnityPlugin/ARM64/Release/HL2RmStreamUnityPlugin
into the folder from step 3.Initialize
function from the DLL, add this statement to one of your Unity scripts:
[DllImport("HL2RmStreamUnityPlugin", EntryPoint = "Initialize", CallingConvention = CallingConvention.StdCall)]
public static extern void InitializeDll();
InitializeDll()
from Unity. An example can be found in UnityHL2RmStreamer.Build Settings -> Player Settings
the following Capabilities are enabled:
package.appxmanifest
in the solution in a text editor to enable the research mode sensors:
Package
:
xmlns:rescap="http://schemas.microsoft.com/appx/manifest/foundation/windows10/restrictedcapabilities"
IgnorableNamespaces="... rescap"
Capabilities
:
<rescap:Capability Name="perceptionSensorsExperimental" />
Package.appxmanifest
Release, ARM64
and deploy to HoloLens2.A simple client written in python for receiving and displaying the frames is available in hololens2_simpleclient.py.