search

cgsaxner / HoloLens2-Unity-ResearchModeStreamer

113 stars 29 forks source link

USB connection for streaming? #19

Closed dogadogan closed 2 years ago

dogadogan commented 2 years ago

When we run deployed the project on HoloLens 2, we experienced is a huge lag. When we tried the approach by @Lancename in #3, we were able to reduce the lag to 3-4 seconds. However, for our applications, we would need it to be <1 second (almost real-time). Is there a way to achieve faster communication? Could this repo be used over USB rather than WiFi, for example?

Thank you for your help! :)

cgsaxner commented 2 years ago

Hi @dogadogan, I have not looked into data transmission over USB, but this Stackoverflow question suggests that TCP over USB is not supported by the HoloLens 2 :(.

I don't really know much about networking, but there are a few things you can look into, like checking your WiFi spectrum for overlapping channels or try setting up a private access point for the HoloLens. Other than the network, the client device might be a limiting factor, e.g., I notice a larger lag when using a notebook compared to a powerful desktop PC. There is also still the possibility of changing the transmission protocol from TCP to UDP, but since UDP is not so reliable, I'm unsure whether this solution is worthwhile for video streaming (since recovering images from unreliable byte data is likely difficult).