MediaStream API should support pan, zoom and tilt

T1mmos commented 9 months ago

⚙ Compilation target

ES2022

⚙ Library

TypeScript 5.3.2

Missing / Incorrect Definition

The interfaces MediaTrackCapabilities and MediaTrackConstraints (MediaTrackConstraintSet) should include the zoom, tilt and pan properties, so control of the camera is accessible via TypeScript.

In pure JavaScript, this works: https://googlechrome.github.io/samples/image-capture/update-camera-zoom.html

In TypeScript, the necessary properties for camera control are unavailable: https://github.com/microsoft/TypeScript/blob/main/src/lib/dom.generated.d.ts (e.g. search for "zoom" -> no results).

Context: a use case where (optical) zoom is required is where a user needs to scan a very tiny QR code (1 cm2). If you move your phone too far away, it's too small to scan; if you move too close, the camera cannot focus on it. A solution here is to move your phone further away but allow the user to zoom which would 1. enable the camera to focus and 2. make the QR code big enough in screen space. I noticed that on my phone, the native photo app zooms by default, no troubles scanning the QR code here.

Sample Code

interface MediaTrackCapabilities {
    aspectRatio?: DoubleRange;
    autoGainControl?: boolean[];
    channelCount?: ULongRange;
    deviceId?: string;
    displaySurface?: string;
    echoCancellation?: boolean[];
    facingMode?: string[];
    frameRate?: DoubleRange;
    groupId?: string;
    height?: ULongRange;
    noiseSuppression?: boolean[];
    sampleRate?: ULongRange;
    sampleSize?: ULongRange;
    width?: ULongRange;
}

interface MediaTrackConstraintSet {
    aspectRatio?: ConstrainDouble;
    autoGainControl?: ConstrainBoolean;
    channelCount?: ConstrainULong;
    deviceId?: ConstrainDOMString;
    displaySurface?: ConstrainDOMString;
    echoCancellation?: ConstrainBoolean;
    facingMode?: ConstrainDOMString;
    frameRate?: ConstrainDouble;
    groupId?: ConstrainDOMString;
    height?: ConstrainULong;
    noiseSuppression?: ConstrainBoolean;
    sampleRate?: ConstrainULong;
    sampleSize?: ConstrainULong;
    width?: ConstrainULong;
}

interface MediaTrackConstraints extends MediaTrackConstraintSet {
    advanced?: MediaTrackConstraintSet[];
}