shaka-project / shaka-player

JavaScript player library / DASH & HLS client / MSE-EME player
Apache License 2.0
7.14k stars 1.34k forks source link

BufferingGoal Not Working with Multi-Period MPEG-DASH in Shaka Player #6354

Closed azoozs closed 7 months ago

azoozs commented 7 months ago

Have you read the Tutorials? Yes

Have you read the FAQ and checked for duplicate open issues? Yes

If the question is related to FairPlay, have you read the tutorial?

N/A

What version of Shaka Player are you using? v4.7.11 and I also tested v2.5.6

What browser and OS are you using? Windows 10 x64 \ Chrome

Please ask your question

I'm using the Shaka Player to play a multi-period MPEG-DASH video, but I'm encountering an issue with the buffering goal. I have an input.mp4 file with a duration of 30 seconds, and I want to split it into segments of 10 seconds each.

Here are the steps I followed:

Step 1: I used ffmpeg to split the video into segments:

ffmpeg -i input.mp4 -c copy -map 0 -segment_time 10 -f segment input_%03d.mp4

This resulted in the following output files:

input_000.mp4
input_001.mp4
input_002.mp4

Step 2: I used MP4Box to dash the segments into periods:

MP4Box -dash 10000 input_000.mp4:period=0 input_001.mp4:period=1 input_002.mp4:period=2 -out out.mpd

This resulted in the following output files:

input_000_dashinit.mp4
input_001_dashinit.mp4
input_002_dashinit.mp4
out.mpd

out.mpd

<?xml version="1.0"?>
<!-- MPD file Generated with GPAC version 2.3-DEV-rev476-gebfad0b5-master at 2024-03-21T09:42:00.505Z -->
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" minBufferTime="PT1.500S" type="static" mediaPresentationDuration="PT0H0M30.077S" maxSegmentDuration="PT0H0M10.136S" profiles="urn:mpeg:dash:profile:full:2011">
 <ProgramInformation moreInformationURL="http://gpac.io">
  <Title>out.mpd generated by GPAC</Title>
 </ProgramInformation>

 <Period id="0" duration="PT0H0M10.346S">
  <AdaptationSet segmentAlignment="true" maxWidth="852" maxHeight="480" maxFrameRate="24000/1001" par="16:9" lang="ja" startWithSAP="1">
   <ContentComponent id="1" contentType="video"/>
   <ContentComponent id="2" contentType="audio"/>
   <Representation id="1" mimeType="video/mp4" codecs="avc1.4D4028,mp4a.40.2" width="852" height="480" frameRate="24000/1001" sar="640:639" bandwidth="989991">
    <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
    <BaseURL>input_000_dashinit.mp4</BaseURL>
    <SegmentList timescale="24000" duration="240000">
     <Initialization range="0-1401"/>
     <SegmentURL mediaRange="1402-1379514" indexRange="1402-1445"/>
    </SegmentList>
   </Representation>
  </AdaptationSet>
 </Period>
 <Period id="1" duration="PT0H0M10.389S">
  <AdaptationSet segmentAlignment="true" maxWidth="852" maxHeight="480" maxFrameRate="24000/1001" par="16:9" lang="ja" startWithSAP="1">
   <ContentComponent id="1" contentType="video"/>
   <ContentComponent id="2" contentType="audio"/>
   <Representation id="2" mimeType="video/mp4" codecs="avc1.4D4028,mp4a.40.2" width="852" height="480" frameRate="24000/1001" sar="640:639" bandwidth="122979">
    <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
    <BaseURL>input_001_dashinit.mp4</BaseURL>
    <SegmentList timescale="24000" duration="240000">
     <Initialization range="0-1449"/>
     <SegmentURL mediaRange="1450-476320" indexRange="1450-1493"/>
     <SegmentURL mediaRange="476321-496706" indexRange="476321-476364"/>
    </SegmentList>
   </Representation>
  </AdaptationSet>
 </Period>
 <Period id="2" duration="PT0H0M9.342S">
  <AdaptationSet segmentAlignment="true" maxWidth="852" maxHeight="480" maxFrameRate="24000/1001" par="16:9" lang="ja" startWithSAP="1">
   <ContentComponent id="1" contentType="video"/>
   <ContentComponent id="2" contentType="audio"/>
   <Representation id="3" mimeType="video/mp4" codecs="avc1.4D4028,mp4a.40.2" width="852" height="480" frameRate="24000/1001" sar="640:639" bandwidth="228267">
    <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
    <BaseURL>input_002_dashinit.mp4</BaseURL>
    <SegmentList timescale="24000" duration="240000">
     <Initialization range="0-1449"/>
     <SegmentURL mediaRange="1450-1014178" indexRange="1450-1493"/>
    </SegmentList>
   </Representation>
  </AdaptationSet>
 </Period>
</MPD>

I then attempted to play the out.mpd file using the Shaka Player in my HTML file. The player is configured with a rebufferingGoal and bufferingGoal of 10. However, the buffering goal does not seem to work with the multi-period MPEG-DASH video.

HTML file:

<!DOCTYPE html>
<html>
  <head>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/shaka-player/4.7.11/shaka-player.compiled.js"></script>
  </head>
  <body>
    <video id="video" width="640" controls></video>
    <script>
        const shaka = window.shaka;
        const video = document.getElementById("video");
        const player = new shaka.Player(video);

        video.addEventListener("play", () => {
            player.configure({
                streaming: {
                rebufferingGoal: 10,
                bufferingGoal: 10
                }
            });
        });

        player.load(
            //"https://storage.googleapis.com/shaka-demo-assets/angel-one/dash.mpd"
            "https://raw.githubusercontent.com/azoozs/shaka-player-multi-period/main/videos/out.mpd"
        );

    </script>
  </body>
</html>

When I use the https://storage.googleapis.com/shaka-demo-assets/angel-one/dash.mpd URL, it works fine. But when I use my own https://raw.githubusercontent.com/azoozs/shaka-player-multi-period/main/videos/out.mpd URL, the buffering goal doesn't seem to work.

My question is: Why doesn't the bufferingGoal: 10 setting work with a multi-period MPEG-DASH video? My aim is to preload the subsequent period before the current one ends, ensuring a seamless transition between periods. Any help would be greatly appreciated. Thank you.

azoozs commented 7 months ago

I’ve discovered that the issue isn’t linked to bufferingGoal. Instead, it’s due to minor gaps between periods. I believe this occurs because MP4Box sets the duration of each period based on the audio stream’s duration, not the video stream’s. You can verify the duration of the video stream using ffprobe:

import subprocess
import json
import datetime

command = f"ffprobe -v quiet -print_format json -show_format -show_streams input_001.mp4"

# Execute the command and get the output
process = subprocess.Popen(command, stdout=subprocess.PIPE, shell=True)
out, err = process.communicate()

# Convert the output to a Python dictionary
output_dict = json.loads(out)

# Extract the video stream duration
for stream in output_dict['streams']:
    if stream['codec_type'] == 'video':
        video_duration = stream['duration']
        print(f"Video Duration: {video_duration}")
    if stream['codec_type'] == 'audio':
        audio_duration = stream['duration']
        print(f"Audio Duration: {audio_duration}")

Afterwards, you’ll need to manually adjust the duration of each period. This should resolve the issue.

Dynamic Solution


from decimal import Decimal
from bs4 import BeautifulSoup
import subprocess
import json
import os
import glob

class PrecisePeriodMPDProducer():
    input_filename = None
    output_mpd_filename = None
    filename_without_extension = None

    def __init__(self, input_filename, output_mpd_filename):
        self.input_filename = input_filename
        self.output_mpd_filename = output_mpd_filename
        self.filename_without_extension = os.path.splitext(self.input_filename)[0]

    def split_video(self, duration=10):
        _, extension = os.path.splitext(self.input_filename)
        if extension.lower() != '.mp4':
            print(f"The file {self.input_filename} is not an .mp4 file.")
            return
        command = f"ffmpeg -i {self.input_filename} -c copy -map 0 -segment_time {duration} -f segment {self.filename_without_extension}_%03d.mp4"  
        process = subprocess.Popen(command, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, shell=True)
        out, err = process.communicate()
        if process.returncode != 0:
            print( f"The split_video command failed with return code: {process.returncode}")

    def get_target_files(self):
        return [f for f in glob.glob("*.mp4") if '_dashinit' not in f and f != self.input_filename]

    def generate_mpd_file(self, dash_duration=10000):
        files_list = self.get_target_files()
        command = "MP4Box" +   f" -dash {dash_duration}"
        for i, file in enumerate(files_list):
            command += f" {file}:period={i}"
        command += f" -out {self.output_mpd_filename}"

        process = subprocess.Popen(command, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, shell=True)
        out, err = process.communicate()
        if process.returncode != 0:
            print( f"The generate_mpd_file command failed with return code: {process.returncode}")

        print("The command was used to create an MPD file: \n" + command ) 

    def convert_seconds_to_mpd_format(self, seconds):
        minutes, seconds = divmod(Decimal(seconds), 60)
        hours, minutes = divmod(minutes, 60)
        mpd_duration = "PT{}H{}M{}S".format(int(hours), int(minutes), seconds)
        return mpd_duration

    def get_video_duration(self, filename):
        command = f"ffprobe -v quiet -print_format json -show_format -show_streams {filename}"
        process = subprocess.Popen(command, stdout=subprocess.PIPE, shell=True)
        out, err = process.communicate()
        output_dict = json.loads(out)
        return Decimal(output_dict['streams'][0]['duration'])

    def execute(self):
        self.split_video(duration=10)
        self.generate_mpd_file(dash_duration=10000)
        with open(f'./{self.output_mpd_filename}', 'r') as mpd_file:
            mpd_data = mpd_file.read()
        soup = BeautifulSoup(mpd_data, 'xml')
        periods = soup.find_all('Period')
        total_video_duration = 0
        for i, period in enumerate(periods):
            base_url = period.find('BaseURL')  
            file_path = os.path.abspath(str(base_url.text).replace("_dashinit",""))
            video_duration = self.get_video_duration(file_path)
            total_video_duration += video_duration
            #print("\n\n" + period.get('duration') + " actual duration: " + str(video_duration) + "\n")
            period['duration'] = self.convert_seconds_to_mpd_format(video_duration) # seconds in ISO 8601 duration format
        #print("Total video duration: " + str(total_video_duration))
        mpd_file.close()
        #print("Total video duration in MPD Format: " + self.convert_seconds_to_mpd_format(total_video_duration))
        with open(f'./{self.output_mpd_filename}', 'w') as mpd_file:
            mpd_file.write(str(soup))

if __name__ == '__main__':
    precise_period_mpd_producer = PrecisePeriodMPDProducer("input.mp4","out.mpd")
    precise_period_mpd_producer.execute()