"tools/vodanalyzer.sh" mpd_proc_cfg error + dash timeline.time mode problem generating .dat

[paulojmcneto]

When trying to generate the .dat files necessary to run dash.timeline.time generated with shaka-packager, the following error appear:

$ ./run_vodanalyzer.sh /var/www/html/dash/vod/orp01-ten999-ch002-timeline-time/Manifest.mpd Traceback (most recent call last): File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/dtc/dash-live-source-simulator/dashlivesim/vodanalyzer/dashanalyzer.py", line 318, in <module> main() File "/home/dtc/dash-live-source-simulator/dashlivesim/vodanalyzer/dashanalyzer.py", line 313, in main dashAnalyzer = DashAnalyzer(mpdFile, verbose) File "/home/dtc/dash-live-source-simulator/dashlivesim/vodanalyzer/dashanalyzer.py", line 86, in __init__ self.mpdProcessor = MpdProcessor(self.mpd_filepath, ) TypeError: __init__() missing 1 required positional argument: 'mpd_proc_cfg'

Accordingly with the syntax, only one parameter (path_to_mpd) is necessary to run "run_vodanalyzer.sh" .

During the error root cause investigation it was possible to see that "run_vodanalyzer.sh" calls dashlivesim\vodanalyzer\dashanalyzer.py where the parameter "mpd_proc_cfg" is mandatory to initialize dashlivesim/dashlib/mpdprocessor.py :

`class MpdProcessor(object): "Process a VoD MPD. Analyze and convert it to a live (dynamic) session."

pylint: disable=no-self-use, too-many-locals, too-many-instance-attributes

def __init__(self, infile, **mpd_proc_cfg**, cfg=None, full_url=None):

`

but in folder dashlivesim/vodanalyzer also exists a mpdprocessor.py with the init function requiring only one parameter. As an attempt to solve the problem on dashanalyzer.py the mpdprocessor import was remapped to:

`

from dashlivesim.dashlib.mpdprocessor import MpdProcessor

from dashlivesim.vodanalyzer.mpdprocessor import MpdProcessor`

With this modification the code started working, but then some more issues appeared.

Case1 - Segment Timeline.Number

When run_vodanalyzer.sh on the following Manifest from a content generated with "shaka-packager", it always ends in "Too much drift in the duration of the segments"

Console Messages: Found 1080p segments 1 - 601 Checking all the media segment durations for deviations. Checking audio with timescale 48000 audio 1 0 289792 First audio segment is 1 starting at time 0.000s Traceback (most recent call last): File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/dtc/dash-live-source-simulator/dashlivesim/vodanalyzer/dashanalyzer.py", line 318, in <module> main() File "/home/dtc/dash-live-source-simulator/dashlivesim/vodanalyzer/dashanalyzer.py", line 314, in main dashAnalyzer.analyze() File "/home/dtc/dash-live-source-simulator/dashlivesim/vodanalyzer/dashanalyzer.py", line 92, in analyze self.checkAndUpdateMediaData() File "/home/dtc/dash-live-source-simulator/dashlivesim/vodanalyzer/dashanalyzer.py", line 252, in checkAndUpdateMediaData raise DashAnalyzerError("Too much drift in the duration of the segments") __main__.DashAnalyzerError: Too much drift in the duration of the segments

Case2 - Segment Timeline.Time

For this case, accordingly with the doc, "run_vodanalyzer.sh" must be used to generate the video and audio ".dat" containing the content time information. Apparently in the dashanalyzer.py is not able to process the timeline.time mode. In a brief code analyses, it falls in the getSegmentRange that should get the first and last segment number, and then an error message raises:

Console Messages:

./run_vodanalyzer.sh /var/www/html/dash/vod/orp01-ten999-ch002-timeline-time/Manifest.mpd Found mediaPresentationDuration = 0s video trackID = 1 Media Regex: $Time$\.m4s Media Reg: re.compile('$Time$\\.m4s') ---- files ---- ['1638400.m4s', '5529600.m4s', '2304000.m4s', ..............., '11673600.m4s'] Match Objects: file 1638400.m4s --- matchObj None Match Objects: file 5529600.m4s --- matchObj None . . . Match Objects: file 11673600.m4s --- matchObj None Traceback (most recent call last): File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/dtc/dash-live-source-simulator/dashlivesim/vodanalyzer/dashanalyzer.py", line 318, in <module> main() File "/home/dtc/dash-live-source-simulator/dashlivesim/vodanalyzer/dashanalyzer.py", line 314, in main dashAnalyzer.analyze() File "/home/dtc/dash-live-source-simulator/dashlivesim/vodanalyzer/dashanalyzer.py", line 91, in analyze self.initMedia() File "/home/dtc/dash-live-source-simulator/dashlivesim/vodanalyzer/dashanalyzer.py", line 122, in initMedia self.getSegmentRange(rep_data) File "/home/dtc/dash-live-source-simulator/dashlivesim/vodanalyzer/dashanalyzer.py", line 163, in getSegmentRange print("Found %s segments %d - %d" % (rep_id, numbers[0], numbers[-1])) IndexError: list index out of range

Can you check if something is being done wrong? Thanks in advance. Paulo Neto

[tobbee]

Thanks for your post.

I also see a problem trying to run the tool's script. The line numbers are bit different, though.

line81: self.mpdProcessor = MpdProcessor(self.mpd_filepath)

I use the latest commit on the develop branch 9ddc490. Which commit/branch did you start from? In any case, I'll try to have a look at the issue and solve it later this week.

Regarding adding your own content, it is rather hard since the system is very picky with what content to use. The start time of the audio and video segments are not allowed to diverge and there must be a point where the segments where the cumulative duration is exactly the same, so that one can make a perfect loop with the same duration for both audio and video. There is also a limitation that the average segment length must be an integral number of seconds. Any NTSC content (29.97fps) content is therefore impossible, or at least very tricky, to add.

The only supported MPD format is live profile with SegmentTemplate with $Number$, although the output format can be SegmentTimeline.

If I would rewrite this whole simulator thingy, I would write it in Go and make it much more forgiving in what input format it would accept, but that is rather hard to retrofit, unfortunately.

There were some discussions in the last DASH-IF f2f meeting about working on a list of wanted improvements for the live simulator and then possibly get some work started if there is some funding. More general input formats should definitely be on that list, and supporting DASH OnDemand format out of the box would be great.

[paulojmcneto]

Many Thanks for your reply Tobbee and for your great contribution.

The version I get when doing "git clone https://github.com/Dash-Industry-Forum/dash-live-source-simulator.git" is the 9ddc490 as well. The difference in the line numbers might be related with some comments I put in the local code during the investigation.

I saw the integer segment duration remark in the wiki, and I’m using 25fps avoiding non integer segment sizes.

In the screenshot below, you can find the $Number$ Manifest I produced using ffmpeg for encoding and shaka packager to package, followed by the .cfg file. The only "manual" optimization I did is in the$RepresentatioID$ and add startWithSAP="1" attribute on Representation element.

with the following .cfg file:

Concerning your comment "The only supported MPD format is live profile with SegmentTemplate with $Number$, although the output format can be SegmentTimeline. ", can you give some tips here, this could help in my case. What do i need to do to achieve segmentTimeLine from segmetNumber? It is not clear for me in the doc how to achieve this, do i need to specify something more in the .cfg or it's an url parameter i need to add in the player url?

Thanks, Paulo Neto

[paulojmcneto]

Hello Tobbie, after my reply to your message, I continued digging in the wiki trying to get timeline.time mode working finally succeeding with testpic4_8s demo asset.

The "run_vodanalyzer" did the job generating the .cfg and .dat files and when i pass the parameter segtimeline_1 i get the expected result:

Now my problem is with my content encoding / packaging that is always rising the error "Too much drift in the duration of the segments", but this is another problem not related with this project.

Your reply was important for me to connect the dots and finally understand how this tool work.

Appart from import issue, this rest is clear for me.

Many Thanks Tobbie. Paulo Neto

[tobbee]

@paulojmcneto Do I understand you correctly that you still have some issues with importing your content?

I think it should be fairly easy to resegment it, so that the audio segments are always ending as close as possible to the corresponding video segments, by tweaking the example code in "https://github.com/edgeware/mp4ff/tree/master/examples/resegmenter"

Currently it writes concatenated segments, but it should be simple to change the code to write one file per segment.

I don't know if and when I would get time to do that, but maybe you can give it a try yourself?

[paulojmcneto]

Hello Tobbie,

I finally succeed to generate the correct encoding / packaging (ffmpeg/shaka) and i'm now able to get number/timeline.number/timeline.time from dashlivesim from what i called my-asset.

I also tried with different duration assets and i get better results when the source vod asset has 1hour. For now i've 2sec segments (1800segment audio / video) and everything seems ok.

Thanks for your help.

[tobbee]

The tool should now be fixed with #113. Sorry for the delay.