Devices with pixel ratio greater than 1

[fred-boink]

Having issues adapting this for mobile, specifically devices with aspect ratios greater than 1 that record in portrait. The results I want is a full screen portrait preview that works on iOS. Any examples or guidance will help!

const renderActiveVideoDevice = (overrideDeviceId?: string) => {
        const canvas = IVSClient.current.getCanvasDimensions();
        const layer = {
            device:
                overrideDeviceId ||
                selectedVideoDevice ||
                videoDevices[0]?.deviceId,
            name: CAM_LAYER_NAME,
            index: 4,
            visible: !camMuted,
            x: 0,
            y: 0,
            width: canvas.width / window.devicePixelRatio,
            height: canvas.height / window.devicePixelRatio,
            type: 'VIDEO'
        };

        addLayer(layer, IVSClient.current);
    };

[slee-aws]

Hello @fred-boink!

While mobile browsers such as iOS Safari are not fully supported in this demo, you can try the following to support portrait video. At a high level, you will need to 1/ Update the SDK config to output a portrait resolution video stream (for example a 1080x1920 video) and 2/ You will need to update the getUserMedia call that your browser makes to request portrait-orientation video from your device camera when it is added to the canvas as a layer. To expand on those two steps further:

1. Update the SDK config Update the streamConfig to use a portrait resolution, such as 1080x1920. The config is initialized in index.js on line 505 and re-applied when the resolution is changed in the settings window on line 404.

The app uses a helper function getConfigFromResolution to set the appropriate streamConfig values for the channel type and resolution selected in settings. However, if you are ok hardcoding those values just to get things working, you can manually set the streamConfig values to force portrait resolution. For example, to output 1080x1920 portrait video:

// index.js line 510

const streamConfig = {
    maxResolution: {
        width: 1080,
        height: 1920,
    },
    maxFramerate: 30,
    maxBitrate: 3500,
};

const IVSClient = IVSBroadcastClient.create({
    streamConfig: streamConfig,
});

2. Update the requested webcam resolution Specifically, you should consider modifying the getUserMedia call in the addVideoLayer function of userLayers.js to request portrait 9:16 video from the device. For example, to request 1080x1920 portrait video:

// useLayers.js line 84

const cameraStream = await navigator.mediaDevices.getUserMedia({
  video: {
    deviceId: { exact: device.deviceId },
    width: {
      ideal: 1080,
      max: 2160,
    },
    height: {
      ideal: 1920,
      max: 3840,
    },
  },
  audio: true,
  aspectRatio: { ideal: 9 / 16 },
  frameRate: 30,
});

Note that getUserMedia does not guarantee that the requested resolution will be returned. You can read more about the getUserMedia API on the following page: https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia.

With the 2 modifications, you should be able to open the app in a browser and see your webcam preview in portrait orientation, provided that your device supports the resolutions defined in getUserMedia:

When hardcoding the portrait stream, note that changes to the resolution in the settings, and the resolution displayed in the top of the app will not be working as expected