search

kixelated / moq-rs

Rust library for Media over QUIC
Apache License 2.0
355 stars 53 forks source link

Formatting the catalog as per common catalog format #174

Open TilsonJoji opened 2 weeks ago

TilsonJoji commented 2 weeks ago

Hello Luke @kixelated , thank you for your work, we ( Gwendal Simon , Tilson Joji ) from Synamedia would like to convey our interest in contributing.

With this PR have attempted to support formatting the catalog as per common catalog format defined in spec https://datatracker.ietf.org/doc/html/draft-wilaw-moq-catalogformat-02

A PR has been requested in moq-js as well to support this change , here is a link to the PR https://github.com/kixelated/moq-js/pull/103

Kindly review and let us know your thoughts.

TilsonJoji commented 2 weeks ago

UT

Pre-code changes catalog format

"tracks": [ { "codec": "avc1.64002A", "container": "mp4", "data_track": "1.m4s", "height": 1080, "init_track": "0.mp4", "kind": "video", "width": 1920 }, { "bit_rate": 129057, "channel_count": 2, "codec": "mp4a.40.2", "container": "mp4", "data_track": "2.m4s", "init_track": "0.mp4", "kind": "audio", "sample_rate": 48000, "sample_size": 16 } ] } Post Code Changes Catalog

[2024-07-08T12:09:09Z INFO moq_pub::media] catalog: { "namespace": "quic.video/watch/bbb", "packaging": "cmaf", "renderGroup": 1, "sequence": 0, "streamingFormat": 1, "streamingFormatVersion": "0.2", "tracks": [ { "data_track": "1.m4s", "init_track": "0.mp4", "name": "video_720p", "selectionParams": { "codec": "avc1.64001F", "height": 720, "width": 1280 } }, { "data_track": "2.m4s", "init_track": "0.mp4", "name": "audio", "selectionParams": { "bit_rate": 125587, "channel_count": 2, "codec": "mp4a.40.2", "sample_rate": 44100, "sample_size": 16 } } ], "version": 1 }

===================================== UT2 ( post address review comments - commit https://github.com/kixelated/moq-rs/pull/174/commits/86a101ddd62856bf884adf0502b0af6822cec9e5 )

[2024-07-11T13:43:13Z INFO moq_pub::media] catalog: { "version": 1, "streamingFormat": 1, "streamingFormatVersion": "0.2", "supportsDeltaUpdates": true, "commonTrackFields": { "namespace": "quic.video/watch/bbb", "packaging": "cmaf", "renderGroup": 1, "altGroup": 1 }, "tracks": [ { "name": "video_720p", "initTrack": "0.mp4", "initData": "1.m4s", "selectionParams": { "codec": "avc1.64001F", "width": 1280, "height": 720 } }, { "name": "audio", "initTrack": "0.mp4", "initData": "2.m4s", "selectionParams": { "codec": "mp4a.40.2", "bitrate": 125587, "samplerate": 44100, "channelConfig": 2 } } ] }

UT3 post commit a0614bd

[2024-07-19T18:14:06Z INFO moq_pub::media] catalog: { "version": 1, "streamingFormat": 1, "streamingFormatVersion": "0.2", "supportsDeltaUpdates": true, "commonTrackFields": { "namespace": "quic.video/watch/bbb", "packaging": "cmaf", "renderGroup": 1, "altGroup": 1 }, "tracks": [ { "namespace": "bbb", "name": "video_720p", "initTrack": "0.mp4", "initData": "1.m4s", "selectionParams": { "codec": "avc1.64001F", "width": 1280, "height": 720 } }, { "namespace": "bbb", "name": "audio", "initTrack": "0.mp4", "initData": "2.m4s", "selectionParams": { "codec": "mp4a.40.2", "bitrate": 125587, "samplerate": 44100, "channelConfig": "2" } } ] }

englishm commented 2 weeks ago

As I mentioned a moment ago, thanks for this, @TilsonJoji (and @gwendalsimon) - the catalog support we have right now is still mostly a placeholder and the format is left over from before the current common catalog spec existed, so I see no reason not to update it both here and in moq-js!

At some point in the future it would be nice to layer in streaming format support as a library and generate catalogs more dynamically, but this is likely the first step either way.

kixelated commented 2 weeks ago

I left some comments on the JS side first. Most of the same comments apply; make sure the fields are correct (ex. camel case) and are in the correct location.

We should implement a proper JSON interface in the Rust code. Here's an initial start from memory:

use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize, Debug)]
pub struct Catalog {
  pub version: String,

  #[serde(rename = "streamingFormat")]
  pub streaming_format: String,

  // etc...

  pub tracks: Vec<Track>,
}

#[derive(Serialize, Deserialize, Debug)]
pub struct Track {
  pub namespace: Option<String>,
  pub name: String,
  pub packaging: TrackPackaging,

  // etc...
}

#[derive(Serialize, Deserialize, Debug)]
pub struct TrackPackaging {
  #[serde(rename = "cmaf")]
  Cmaf

  #[serde(rename = "loc")]
  Loc
}

#[derive(Serialize, Deserialize, Debug, Default)]
pub struct Selection {
  pub codec: Option<String>,
  pub mimeType: Option<String>,
  pub frameRate: Option<u64>,

  // etc...
}

And then to actually use this type you would:

let catalog = Catalog {
  version: "1".to_string(),
  // ...etc
  tracks: vec![
    Track {
      name: "video".to_string(),
      packaging: TrackPackaging::Cmaf,
    },
    Track {
      name: "audio".to_string(),
      packaging: TrackPackaging::Cmaf,
    }
  ],
};

let output = serde_json::to_string(catalog)?;

// Or to decode
let catalog: Catalog = serde_json::from_str(&output)?;
kixelated commented 2 weeks ago

Oh and we should make a moq-catalog or moq-warp crate for this eventually.

TilsonJoji commented 1 week ago

Thank you @englishm.

Thank you @kixelated for the detailed review and the feedback. I will address the comments and update the PR.

TilsonJoji commented 1 week ago

Hello Luke @kixelated , Mike @englishm , i have attempted to address the comments and followed the latest version of the spec , kindly review and let me know your thoughts.

TilsonJoji commented 3 days ago

Hello Luke @kixelated , Mike @englishm , please be informed have fixed the workflow build warnings and have updated the code to work with latest commit 7baad29 as on 19th July'24 in moq-js PR. Kindly have a look as time allows and let me know your thoughts.

UT1 :

UT2:

  1. Access URL https://ip:4321/publish?server=IP:4443
  2. Click on Camera
  3. Click on Go live and then on share
  4. Watch the stream at https://ip:4321/watch/82dd0119-de7e-4f5c-8fb1-a4768a29fb5b?server=IP:4443
kixelated commented 1 day ago

@TilsonJoji same deal, I managed to snag some time while my wife is asleep to fix the PR inline. I didn't make many changes, mostly just moved the catalog stuff to a new moq-catalog crate that we can publish.

Untested again, lemme know when everything is stable on your end and I'll test both the JS and Rust side.

I started to upgrade moq-sub but ugh that code needs some work. I already rewrote it on another branch so I'm going to get that merged instead.

TilsonJoji commented 1 day ago

Hello Luke @kixelated , thank you for the commit 73bad56 f05c00e

For 73bad56 f05c00e , moq-js required a small change with regards to accessing namespace from catalog , i have tried to incorporate the fix with this commit b8d5a80/511ad99 in PR https://github.com/kixelated/moq-js/pull/103

Performed the below UTs and it works.

UT1 :

./dev/relay ./dev/pub Access stream at URL https://ip:4321/watch/bbb?server=IP:4443 ( chrome browser )

UT2:

Access URL https://ip:4321/publish?server=IP:4443 Click on Camera Click on Go live and then on share Watch the stream at https://ip:4321/watch/82dd0119-de7e-4f5c-8fb1-a4768a29fb5b?server=IP:4443