search

jdibenes / hl2ss

HoloLens 2 Sensor Streaming. Real-time streaming of HoloLens 2 sensor data over WiFi. Research Mode and External USB-C A/V supported.
Other
205 stars 51 forks source link

Simultaneous capturing of video locally on HL2 while streaming #143

Open anjelomerte opened 4 days ago

anjelomerte commented 4 days ago

Hey Juan Carlos, thanks for engaging with the community on your lovely project! Since a lot has changed since the last time I worked with your project, I got two things on my mind:

  1. Is it possible to record video footage locally onto HL2 while streaming it to an external PC?
  2. For a server-controlled streaming workflow, is it still the best way to poll for server input or are there any new possibilities I am not aware of?

Best regards!

jdibenes commented 4 days ago

Hi,

  1. No. Support for that would have to be implemented in the C++ code.
  2. Yes, streaming is still controlled by the client.
anjelomerte commented 3 days ago

Regarding 1, is it simply not possible with the hl2ss plugin or is it inherently not possible to record while also stream? In other words, if I were to implement video recording in Unity/C# while the hl2ss streaming plugin is running, would that cause problems? If I had a HL2 with me I would simply test it but currently I don't, that's why I am asking.

jdibenes commented 2 days ago

It should be possible to record while streaming with some caveats. For example, if the hl2ss plugin is streaming the PV camera then the Unity/C# code must access the camera in shared mode. Otherwise, the Unity/C# code will take over the camera and streaming will stop (the hl2ss client will be disconnected). Conversely, if the Unity/C# code is using the camera and then some hl2ss client requests the camera stream it must do so in shared mode. Otherwise, hl2ss will take control of the camera and the Unity/C# stream will be closed. So some synchronization between server and client is necessary. For example, the Unity/C# code could initialize the camera to the desired resolution and framerate and keep the camera running for the duration of the app, while the hl2ss client always connects in shared mode (in shared mode, resolution and framerate cannot be changed).

anjelomerte commented 1 day ago

Thank you for your input! Your proposed solution sounds feasible. So would you agree that this is the best approach to preview captured video footage on the HL2? I mean capturing by the client is possible but I see no straightforward way of getting that captured footage back onto the HL2 or am I missing sth?

jdibenes commented 5 hours ago

Yes, I think so. But just to preview the video captured by the client, I would add callbacks to the C++ code to detect when a client connects/disconnects and forward raw frames to the C# code to avoid managing two streams.