w3c / mediacapture-record

MediaStream Recording
https://w3c.github.io/mediacapture-record/
Other
104 stars 21 forks source link

Add method that allows user specification of which tracks go with what number in the container-dependent ordering #179

Open guest271314 opened 4 years ago

guest271314 commented 4 years ago

I think the Right Solution is to make MediaRecorder have a method that allows you to specify which tracks go with what number in the container-dependent ordering.

This change means a method such as

MediaRecorder.setTrackOrder({track: MediaRecorder.stream.getAudioTracks()[0], orderId:0});
MediaRecorder.setTrackOrder({track: MediaRecorder.stream.getVideoTracks()[0], orderId:1});

To the degree deemed to be controlling or relevant at this specification, track order language is found at https://w3c.github.io/mediacapture-main/

The relative order of the tracks in the set is User Agent defined and the API will never put any requirements on the order.

However, after filing this issue at that specification, was referred to the mediacapture-record specification as owner of the context https://github.com/w3c/mediacapture-main/issues/611.

If a MediaStream contains both audio and video then the order described at the first sentence should be applied.

The reason for the change is for post-record editing, which becomes tedious when the order of the tracks is arbitrary.

Consider two webm files output by MediaRecorder at Chromium

$ ./mkvmerge -J 1.webm
{
  "attachments": [],
  "chapters": [],
  "container": {
    "properties": {
      "container_type": 17,
      "is_providing_timestamps": true,
      "muxing_application": "Chrome",
      "writing_application": "Chrome"
    },
    "recognized": true,
    "supported": true,
    "type": "Matroska"
  },
  "errors": [],
  "file_name": "1.webm",
  "global_tags": [],
  "identification_format_version": 12,
  "track_tags": [],
  "tracks": [
    {
      "codec": "Opus",
      "id": 0,
      "properties": {
        "audio_bits_per_sample": 32,
        "audio_channels": 1,
        "audio_sampling_frequency": 48000,
        "codec_id": "A_OPUS",
        "codec_private_data": "4f707573486561640101000080bb0000000000",
        "codec_private_length": 19,
        "default_track": true,
        "enabled_track": true,
        "forced_track": false,
        "language": "eng",
        "minimum_timestamp": 119000000,
        "number": 1,
        "uid": 40889044785409303
      },
      "type": "audio"
    },
    {
      "codec": "VP8",
      "id": 1,
      "properties": {
        "codec_id": "V_VP8",
        "codec_private_length": 0,
        "default_track": true,
        "display_dimensions": "1046x588",
        "display_unit": 0,
        "enabled_track": true,
        "forced_track": false,
        "language": "eng",
        "minimum_timestamp": 0,
        "number": 2,
        "pixel_dimensions": "1046x588",
        "uid": 17593832491204014
      },
      "type": "video"
    }
  ],
  "warnings": []
}

$ /mkvmerge -J 2.webm
{
  "attachments": [],
  "chapters": [],
  "container": {
    "properties": {
      "container_type": 17,
      "is_providing_timestamps": true,
      "muxing_application": "Chrome",
      "writing_application": "Chrome"
    },
    "recognized": true,
    "supported": true,
    "type": "Matroska"
  },
  "errors": [],
  "file_name": "2.webm",
  "global_tags": [],
  "identification_format_version": 12,
  "track_tags": [],
  "tracks": [
    {
      "codec": "VP8",
      "id": 0,
      "properties": {
        "codec_id": "V_VP8",
        "codec_private_length": 0,
        "default_track": true,
        "display_dimensions": "320x240",
        "display_unit": 0,
        "enabled_track": true,
        "forced_track": false,
        "language": "eng",
        "minimum_timestamp": 0,
        "number": 1,
        "pixel_dimensions": "320x240",
        "uid": 33743719209670163
      },
      "type": "video"
    },
    {
      "codec": "Opus",
      "id": 1,
      "properties": {
        "audio_bits_per_sample": 32,
        "audio_channels": 2,
        "audio_sampling_frequency": 48000,
        "codec_id": "A_OPUS",
        "codec_private_data": "4f707573486561640102000080bb0000000000",
        "codec_private_length": 19,
        "default_track": true,
        "enabled_track": true,
        "forced_track": false,
        "language": "eng",
        "minimum_timestamp": 8000000,
        "number": 2,
        "uid": 56209065175931709
      },
      "type": "audio"
    }
  ],
  "warnings": []
}

Not only does the number of channels need to be adjusted, though also, to merge the two files the developer needs to map which next track to append to the current track. For only two tracks that can be simple, for multiple tracks in arbitrary order, the task can become meticulous.

This change will also provide uniformity for all implementations, for example, the ability to merge a WebM file output by Chromium/Chrome and a webm file output by Firefox, setting aside for the moment that Chrome sets audio_sampling_frequency to 48000, while Firefox sets the property to 44100.