Support SegmentList with mediaRange in DASH

ctoshni commented 6 years ago

Hi, i'm using latest Shaka version ; I have two questions:

Is it possible to generate "mediaRange" with byte-offset using shaka-packager like below?
If so then what are the options I need to use in the command?

<Representation id="1" mimeType="audio/mp4" codecs="mp4a.40.2" audioSamplingRate="44100" startWithSAP="1" bandwidth="263127">
    <BaseURL>256kps_dash.mp4</BaseURL>
    <SegmentList timescale="44100" duration="441000">
      <SegmentURL mediaRange="960-329352" indexRange="960-1003"/>
      <SegmentURL mediaRange="329353-657797" indexRange="329353-329396"/>
      <SegmentURL mediaRange="657798-986186" indexRange="657798-657841"/>
      <SegmentURL mediaRange="986187-1314578" indexRange="986187-986230"/>
      <SegmentURL mediaRange="1314579-1642769" indexRange="1314579-1314622"/>
      <SegmentURL mediaRange="1642770-1971091" indexRange="1642770-1642813"/>

Thanks

kqyang commented 6 years ago

No, we do not support SegmentList nor mediaRange within the SegmentList.

Is there any reason you want to use SegmentList?

We support SegmentBase with DASH on-demand profile and SegmentTemplate with DASH live profile. Manifest with SegmentBase or SegmentTemplate are usually more compact than manifest with SegmentList.

ctoshni commented 6 years ago

Is there any reason you want to use SegmentList?

I want to use single segmented file with DASH having byte offset for each segment.

SegmentBase only contains information about sidx. Due to this it becomes responsibility of the player to read the sidx and get the offset of segments. Eg of such MPD.

<Representation id="0" bandwidth="74236" codecs="opus" mimeType="audio/mp4" audioSamplingRate="48000">
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
        <BaseURL>lo_dash.mp4</BaseURL>
        <SegmentBase indexRange="784-1007" timescale="48000">
          <Initialization range="0-783"/>
        </SegmentBase>
      </Representation>

I want to generate a MPD which contains byte offset for all the segments.

We support SegmentBase with DASH on-demand profile and SegmentTemplate with DASH live profile. Manifest with SegmentBase or SegmentTemplate are usually more compact than manifest with SegmentList.

If it is not possible to have mediaRange within SegmentList so is it possible to have it inside SegmentTemplate or SegmentBase. Eg like below or something similar?


      < SegmentTemplate mediaRange="960-329352" indexRange="960-1003"/>
      < SegmentTemplate mediaRange="329353-657797" indexRange="329353-329396"/>
      < SegmentTemplate mediaRange="657798-986186" indexRange="657798-657841"/>
      < SegmentTemplate mediaRange="986187-1314578" indexRange="986187-986230"/>
      < SegmentTemplate mediaRange="1314579-1642769" indexRange="1314579-1314622"/>
      < SegmentTemplate mediaRange="1642770-1971091" indexRange="1642770-1642813"/>

or something similar?

If yes, then what all the options I need to give?

kqyang commented 6 years ago

SegmentBase only contains information about sidx. Due to this it becomes responsibility of the player to read the sidx and get the offset of segments. Eg of such MPD.

Yes, but all DASH players I know of supporting sidx parsing.

If it is not possible to have mediaRange within SegmentList so is it possible to have it inside SegmentTemplate or SegmentBase. Eg like below or something similar?

No, that is not allowed by the DASH spec as mediaRange is only allowed in SegmentURL element and SegmentURL is only allowed in SegmentList.

Not sure what is your use case. If what you want is to avoid parsing sidx and you don't mind using HLS, then you can generate HLS playlists using Shaka Packager.

ctoshni commented 6 years ago

Thanks! My use-case is to generate mediaRange into mpd file for a single segmented file as we want to avoid sidx parsing on client side.

kqyang commented 6 years ago

We do not have plan to support SegmentList right now. You may consider extending Shaka Packager yourself to support it.

ajaymore commented 5 years ago

Hi, Thank you for a wonderful Open Source Project.

I am trying to use google cloud storage to store my content, which is created using Shaka Packager. Currently I use registerRequestFilter to get a gcloud storage Signed Url and then use the same to load content.

With a structure as below

<SegmentTemplate timescale="12800" initialization="h264_720p/init.mp4" media="h264_720p/$Number$.m4s" startNumber="1">
          <SegmentTimeline>
            <S t="0" d="128000" r="1"/>
            <S t="256000" d="65536"/>
            <S t="321536" d="128000" r="7"/>
            <S t="1345536" d="51712"/>
            <S t="1397248" d="119296"/>
            <S t="1516544" d="128000" r="1"/>
            <S t="1772544" d="197120"/>
            <S t="1969664" d="128000" r="1"/>
            <S t="2225664" d="66560"/>
          </SegmentTimeline>
        </SegmentTemplate>

Would it be possible to modify and directly insert those signed urls here. I am asking this because I am still quite new to the mpd specification. So even if Shaka Packager doesn't support Absolute URLs is it possible for me to modify the static mpd file and insert my own URLs?

kqyang commented 5 years ago

@ajaymore I don't know of anyway to do it in packager without using SegmentList as every segment is signed with different signatures. But it might be possible to do it in the player, e.g. by signing the segment url on the fly during playback. @joeyparrish Any thoughts on this?

joeyparrish commented 5 years ago

If you can use a request filter to get a signed URL from cloud storage in the client, you could do that for individual segments as well as the manifest. You can see the request type (shaka.net.NetworkingEngine.RequestType.SEGMENT, shaka.net.NetworkingEngine.RequestType.MANIFEST, etc.) and inspect/modify the URLs (request.uris, which is an array) as you like.

Does that help?

ajaymore commented 5 years ago

@kqyang, @joeyparrish currently I am using express server to proxy the requests

router.get('/video**', (req, res) => {
  const key = req.url.replace('/video', ''); // extract the file key from request sent by shaka player
  var file = req.bucket.file(key); // instance of file in google cloud storage 

  var config = {
    action: 'read',
    expires: '03-17-2025'
  };

  file.getSignedUrl(config, function(err, url) {
    if (err) {
      console.error(err);
      res.status(400).json(err);
    }

    // make request to google cloud server and and directly send the response to shaka player
    require('request')
      .get(url)
      .pipe(res);
  });
});

The limitation with above approach is unnecessary bandwidth consumption of my web server.

The second approach that I have tried is getting the signed url using shaka player's registerRequestFilter. This approach works just fine.

player
          .getNetworkingEngine()
          .registerRequestFilter(function(type: any, request: any) {
            // console.log(type, request.uris[0]);
            if (
              type ===
              (window as any).shaka.net.NetworkingEngine.RequestType.MANIFEST
            ) {
              //   return fetch(manifestUri)
              //     .then(res => res.json())
              //     .then(data => {
              //       request.uris[0] = data;
              //     });
            } else {
              return fetch(`/api/gcloud-segment?path=${request.uris[0]}`)
                .then(res => res.json())
                .then(data => {
                  request.uris[0] = data;
                });
            }
          });

My concern is that on native mobile phones where I do not use Shaka Player, my second solution is useless.

I am not expecting any new feature to be added to shaka packager. However if I were to parse the current .mpd file in node and transform it, how should I go about doing it? For Example how do I convert something like below from shaka packager to

<Representation id="1" bandwidth="567106" codecs="avc1.64001e" mimeType="video/mp4" sar="640:639" width="852" height="480">
        <SegmentTemplate timescale="12800" initialization="h264_480p/init.mp4" media="h264_480p/$Number$.m4s" startNumber="1">
          <SegmentTimeline>
            <S t="0" d="128000" r="1"/>
            <S t="256000" d="70656"/>
            <S t="326656" d="128000" r="7"/>
            <S t="1350656" d="46592"/>
            <S t="1397248" d="119296"/>
            <S t="1516544" d="128000" r="1"/>
            <S t="1772544" d="178688"/>
            <S t="1951232" d="128000" r="1"/>
            <S t="2207232" d="84992"/>
          </SegmentTimeline>
        </SegmentTemplate>
      </Representation>

To something like below?

<Representation id="1" mimeType="video/mp4" codecs="avc1.64002A,mp4a.40.2" width="1920" height="1080" frameRate="25" sar="1:1" audioSamplingRate="48000" bandwidth="3333259">
    <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
    <SegmentList timescale="1000" duration="4000">
     <Initialization sourceURL="segment_init.mp4"/>
     <SegmentURL media="segment_1.m4s"/>
     <SegmentURL media="segment_2.m4s"/>
     <SegmentURL media="segment_3.m4s"/>
     <SegmentURL media="segment_4.m4s"/>
     <SegmentURL media="segment_5.m4s"/>
     <SegmentURL media="segment_6.m4s"/>
     <SegmentURL media="segment_7.m4s"/>
     <SegmentURL media="segment_8.m4s"/>
     <SegmentURL media="segment_9.m4s"/>
     <SegmentURL media="segment_10.m4s"/>
     <SegmentURL media="segment_11.m4s"/>
     <SegmentURL media="segment_12.m4s"/>
     <SegmentURL media="segment_13.m4s"/>
     <SegmentURL media="segment_14.m4s"/>
     <SegmentURL media="segment_15.m4s"/>
     <SegmentURL media="segment_16.m4s"/>
     <SegmentURL media="segment_17.m4s"/>
     <SegmentURL media="segment_18.m4s"/>
     <SegmentURL media="segment_19.m4s"/>
     <SegmentURL media="segment_20.m4s"/>
     <SegmentURL media="segment_21.m4s"/>
     <SegmentURL media="segment_22.m4s"/>
    </SegmentList>
   </Representation>

ajaymore commented 5 years ago

If you can use a request filter to get a signed URL from cloud storage in the client, you could do that for individual segments as well as the manifest. You can see the request type (shaka.net.NetworkingEngine.RequestType.SEGMENT, shaka.net.NetworkingEngine.RequestType.MANIFEST, etc.) and inspect/modify the URLs (request.uris, which is an array) as you like.

Does that help?

Yes I am doing that and it works like a charm. I posted another limitation of mine in above comment.

kqyang commented 5 years ago

@ajaymore

However if I were to parse the current .mpd file in node and transform it, how should I go about doing it?

Yes, you can certainly parse and transform it outside of packager. See ISO/IEC 23009-1:2014 section 5.3.9.4.4 on how you can construct segment URL from the template.

Here is a basic idea:

media="h264_480p/$Number$.m4s" startNumber="1"

This gives the URL templates and the starting number.

    <S t="0" d="128000" r="1"/>

In the S element, r is the repeating number. A r=1 means it is repeated once, i.e. there are two segments.

For something like below, let's assume the start number is 1000:

    <S t="0" d="128000" r="1"/>
    <S t="256000" d="70656"/>
    <S t="326656" d="128000" r="7"/>

It can be expanded to:

    <!-- expanded from S@t=0 -->
    <SegmentURL media="h264_480p/1000.m4s"/>
    <SegmentURL media="h264_480p/1001.m4s"/>
    <!-- expanded from S@t=256000 -->
    <SegmentURL media="h264_480p/1002.m4s"/>
    <!-- expanded from S@t=326656 -->
    <SegmentURL media="h264_480p/1003.m4s"/>
    <SegmentURL media="h264_480p/1004.m4s"/>
    <SegmentURL media="h264_480p/1005.m4s"/>
    <SegmentURL media="h264_480p/1006.m4s"/>
    <SegmentURL media="h264_480p/1007.m4s"/>
    <SegmentURL media="h264_480p/1008.m4s"/>
    <SegmentURL media="h264_480p/1009.m4s"/>
    <SegmentURL media="h264_480p/1010.m4s"/>

Note that to use SegmentList, the segments should have roughly the same duration. To achieve that, the streams should have constant GOPs and the segment duration should be set to multiple of GOP size.

Does that answer your question?

ajaymore commented 5 years ago

Thank you @kqyang. It does answer my question. I will have to try out this approach to se how it works.

samidhtalsania commented 3 years ago

Hi @kqyang, I am planning to implement this feature. Do you have any thoughts on how to achieve this? I am still going through the code base and will post here on how I plan to do it.

kqyang commented 3 years ago

@samidhtalsania Great. We already have segment list and byte-range support in Shaka Packager for HLS. I think we can extend it to support SegmentList for DASH.

See below for the suggested work items:

Update RepresentationXmlNode class to support SegmentURL list generation. We may add a new function AddSegmentList or update the existing functions.
Update Representation class. Two notable functions needed to be updated are:
- Representation::GetXml to support SegmentList generation
- Representation::AddSegmentInfo should take the segment file name and start byte offset too. You may take a look at the MediaPlaylist::AddSegment implementation.
Update MpdNotifyMuxerListener class and MpdNotifyMuxerListener::OnMediaEnd function to pass over the byte ranges. You may take a look at how it is implemented for HLS: HlsNotifyMuxerListener::OnMediaEnd.
Update SimpleMpdNotifier class accordingly.

We should support both VOD and Live profiles. Preferably it should be done in multiple PRs for easy code review.

@samidhtalsania Let me know if you have any questions.

shaka-project / shaka-packager

Support SegmentList with mediaRange in DASH #455