search

microsoft / MixedReality-WebRTC

MixedReality-WebRTC is a collection of components to help mixed reality app developers integrate audio and video real-time communication into their application and improve their collaborative experience
https://microsoft.github.io/MixedReality-WebRTC/
MIT License
908 stars 282 forks source link

Camera constraints on HoloLens 1 not working #563

Open dl4mmers opened 3 years ago

dl4mmers commented 3 years ago

Describe the bug When setting video capture constraints in WebcamSource manually in a HoloLens 1 build, no video is shown at the remote peer and the application crashes when activating Mixed Reality Capture. When capture mode is set to Automatic, everything works fine on HoloLens 1. On HoloLens 2 there is no problem with setting video capture constraints manually, video is always showing up on the remote peer.

All constraints we are setting are based on this Microsoft documentation: https://docs.microsoft.com/en-us/windows/mixed-reality/locatable-camera

I did then take a look into WebcamSource script and manually set the constraints according to automatic mode in WebcamSource script, but the video still didn't show up. I have also tried to only set one of the values, like width = 1280 and set the other values to zero, because the comment in WebcamSource script says "Avoid constraining the framerate".

I was unsure which video profile I should set for HoloLens 1, because as far as I understand, there are no camera profiles for HoloLens 1. I set it to VideoProfileKind.VideoRecording like in WebcamSource script automatic mode, but also tried to leave it blank.

I tried different combinations of constraints, with and without setting VideoProfileKind, but only got the video showing up at remote peer with WebcamSource capture format set to Automatic.

To Reproduce Steps to reproduce the behavior:

  1. Set WebcamSource capture format to Manual and set constraints to one of the following combinations:
    • width = 1280, height = 720, framerate = 30, video profile kind = Video Conferencing
    • width = 1280, height = any, framerate = any, video profile kind = any
  2. Deploy on HoloLens 1 and establish a call
  3. See that there is no video shown at remote peer
  4. Enable Mixed Reality Capture and see app is crashing

Expected behavior HoloLens 1 video should show up at remote peer when call is established and WebcamSource mode is set to Manual with correct constraints based on this documentation: https://docs.microsoft.com/en-us/windows/mixed-reality/locatable-camera.

Environment

fibann commented 3 years ago

VideoConferencing is empty on HL1, 1280x720@30 is in VideoRecording instead. Can you check if (1280, 720, 30, VideoRecording) and (1280, any, any, VideoRecording) work?

Passing an unspecified video profile should work, this looks like a bug.

dl4mmers commented 3 years ago

Sorry for the late response.

(1280, 720, 30, VideoRecording) and (1280, any, any, VideoRecording) both don't work, no video is shown at remote peer and the app crashes when enabling MRC.