Add a track attribute for 'type'

[wilaw]

There are a number of use-cases in which it is necessary for the streaming application to be able to declare the 'type' of a track. For example, a 'data channel', or a 'timeline' track. The attribute can be a String. We may want to set a max length. This field is optional. The application using the catalog format would define the contents of this field.

"tracks":[
    {
      "name": "game-instructions",
      "type": "datachannel"
    },
    {
      "name": "media-timeline",
      "type": "timeline"
    },
    {
      "name": "hd",
      "selectionParams": {"codec":"av01","width":1920,"height":1080,
      "bitrate":5000000,"framerate":30},
      "altGroup":1
    },
    {
      "name": "md",
      "selectionParams": {"codec":"av01","width":720,"height":640,
      "bitrate":3000000,"framerate":30},
      "altGroup":1
    },
    {
      "name": "sd",
      "selectionParams": {"codec":"av01","width":192,"height":144,
      "bitrate":500000,"framerate":30},
      "altGroup":1
    },
    {
      "name": "audio",
      "selectionParams":{"codec":"opus","samplerate":48000,"channelConfig":"2",
      "bitrate":32000}
    }
   ]

[jiuhaizhang]

Thank you Will. It's works for my cloud gaming application with datachannel. But Should video track and audio track also declare type='video' or type='audio'? When I receive a track, I don't know whether to process as video or audio because the mime type is not required.

//When catalog received
if (selection_params.mime_type) {
    if (strncmp(selection_params.mime_type, "video", strlen("video")) == 0) {
        track_type = TRACK_VIDEO;
        // notify user can subscribe video track
    } else if (strncmp(selection_params.mime_type, "audio", strlen("audio")) == 0) {
       track_type = TRACK_AUDIO;
       // notify user can subscribe audio track
    }
}
if (selection_params.samplerate > 0) {
    track_type = TRACK_AUDIO;
}

//When Object received
track = find_track_by_alias(track_alias)
if (track->track_type == TRACK_VIDEO) {
    //Decode video 
} else if (track->track_type == TRACK_AUDIO) {
    //Decode audio
}

[wilaw]

Ref “When I receive a track, I don't know whether to process as video or audio because the mime type is not required.”

The track fields in the catalog are optional. That’s because the catalog is like a base class that can be used and extended by your cloud gaming application. So your cloud gaming spec should create a rule that says that for your cloud gaming catalogs, the mime-type attribute should always be present. That’s what I intend to do with the WARP draft, which is an example of a streaming application. Additionally, you could create rules that the video tracks must declare the type to “video”. We may add a similar rule to base catalog, although I’d prefer to keep it more generic and allow the applications to define the values.