Integrating a Custom Holographic Remote Processing Player Application with Hololens2 Sensor Data Stream Server

[BroPinLove]

Dear jdibenes,

Hello!

First, please allow me to express my sincere respect to you. I am currently attempting to integrate the holographic remote processing player application (GitHub repository link) with the Hololens2 sensor data stream C++ server (GitHub repository link). My goal is to project the Unity project scene content from the PC to HoloLens2 while simultaneously sending the HoloLens sensor data stream to a Python client. I referred to this closed issue post (GitHub Issue #105) and attempted the integration based on the discussion therein. However, due to my limited experience with C++, I encountered some issues 。

I downloaded the plugin_1.0.28.0.zip file from Link and added the hl2ss.dll, Microsoft.MixedReality.EyeTracking.dll, Microsoft.MixedReality.SceneUnderstanding.dll, as well as the related .h and .lib files to the root directory of the SamplePlayer (Universal Windows) C++ program.

By right-clicking the project > Add > Existing Item, I added the three .dll files to the project. Then, I right-clicked each .dll file > Properties > set Content to "Yes".By right-clicking the project > Add > Existing Item In the SamplePlayerMain.cpp code, I called the InitializeStreams() method as shown in the image below:

I deployed this C++ project to my HoloLens2 via WiFi and ran the application. On the PC, I ran the Python client program client_stream_pv.py (with HoloLens2 and PC on the same WiFi network). However, the Python client did not respond. I have tried multiple times, but the result is always the same. I hope you can help me with the following questions: Are there any steps or errors I may have overlooked or made when integrating and calling the sensor data stream? Is there more detailed documentation or examples that I can refer to? Thank you very much for your time and assistance. Yours sincerely,

Pin

[jdibenes]

Hello, Check that the "Sample Holographic Remoting Player" app has access to the camera (Settings > Privacy > Camera). Also, does the hl2ss app work? If not, your HoloLens OS may be outdated or there may be an issue with your network.

[BroPinLove]

Thank you very much for your response; I have resolved the issue following your advice. Due to my busy schedule last week, I am only now sharing this good news. Recently, I made some new attempts: I changed the parameter of InitializeStreams() to 3U and was able to obtain depth data from HoloLens2 on the Python side, but I was unable to access the color data. If I want to simultaneously acquire both color and depth data, what should I do? I would like to run sample_viewer_si.py on the Python side.

[jdibenes]

Try enabling all streams. Just pass 0xFFFFFFFF to InitializeStreams.

[BroPinLove]

Awesome, everything runs smoothly thank you so much

[Orign-al]

Hi。These are amazing projects,.And i an trying to do what you have done, but once i put the InitializeStreams() in the place where you put , my vs will report

LNK2019 __imp_InitializeStreams， "public: void __cdecl SamplePlayerMain::SetWindow(struct winrt::Windows::UI::Core::CoreWindow const &)" (?SetWindow@SamplePlayerMain@@QEAAXAEBUCoreWindow@Core@UI@Windows@winrt@@@z) qoute the SamplePlayer C:\Users\29001\Downloads\MixedReality-HolographicRemoting-Samples-main\MixedReality-HolographicRemoting-Samples-main\player\sample\SamplePlayerMain.obj .

I cant fix this .Could you help me to fix this or send your project. Thanks ! !

IF my tone makes you uncomfortable , i apologize for it here because i am not a native English user. Thanks again! ! ！ ! @BroPinLove @jdibenes @Walidkilas

[jdibenes]

Hello, You have to link your project with hl2ss.lib (from the releases zip file).

1. In VS in the Solution Explorer, right click your project and select Properties.
2. In Configuration Properties > Linker > General > Additional Library Directories add the folder containing hl2ss.lib.
3. In Configuration Properties > Linker > Input > Additional Dependencies add hl2ss.lib to the list.

[Orign-al]

OHHHHH！It works !!!!! And I have found that the lib in the plugin_1.0.28.0.zip seems cant work.It can be deployed into the hololens but cant play. So, I use the lib in the plugin_unity_1.0.29.0.zip it takes me lots of time to find where the lib is. Thank you very much!!!What an amazing project ,help me a lot.