axiomatic-systems / Bento4

Full-featured MP4 format, MPEG DASH, HLS, CMAF SDK and tools
http://www.bento4.com
1.97k stars 479 forks source link

Sync segment duration of audio and video stream with mp4dash #790

Open bleibdirtroy opened 1 year ago

bleibdirtroy commented 1 year ago

Hey, I'm having trouble generating a hls stream with equal segment duration for audio and video.

My first step is to force keyframes on audio and video files every 2 seconds with ffmpeg.

ffmpeg -t $duration -i original.mp4 -c:v libx264 -an -vf "scale=-1:1080" -b:v 6M -force_key_frames 'expr:gte(t,n_forced*'"$keyframe_interval"')' -r $fps 1080p.mp4
ffmpeg -t $duration -i original.mp4 -c:a aac -vn -force_key_frames 'expr:gte(t,n_forced*'"$keyframe_interval"')' -r $fps audio_1.mp4

Then I fragment those two files with mp4fragment.

mp4fragment --fragment-duration 6000 1080p.mp4 f-1080p.mp4
mp4fragment --fragment-duration 6000  audio_1.mp4 f-audio_1.mp4

And finally I create my hls live stream with mp4dash.

mp4dash -o stream --hls --use-segment-timeline f-1080p.mp4 f-audio_1.mp4

This generates the following two playlists (and other stuff of but not important here) Audio Playlist

#EXTM3U
# Created with Bento4 mp4-dash.py, VERSION=2.0.0-639
#
#EXT-X-VERSION:6
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-TARGETDURATION:7
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-MAP:URI="init.mp4"
#EXTINF:5.990748299319728,
seg-1.m4s
#EXTINF:6.013968253968254,
seg-2.m4s
#EXTINF:5.990748299319728,
seg-3.m4s
....

Video Playlist

# Created with Bento4 mp4-dash.py, VERSION=2.0.0-639
#
#EXT-X-VERSION:6
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-MAP:URI="init.mp4"
#EXTINF:6.0,
seg-1.m4s
#EXTINF:6.0,
seg-2.m4s
#EXTINF:6.0,
seg-3.m4s
....

The problem is, that:

  1. Audio and Video segments in the created playlists have not the same length. My goal is to have exactly #EXTINF:6.0, on every segment for video and audio.
  2. Can I limit the or fix the digits for the segment duration. Meaning e.g. max 3 digits as segment length so #EXTINF:6.013968253968254, turns into #EXTINF:6.014,?

The first point would make the second obsolete. And I guess point 2 is not fixed by rounding the number.

max-arnold commented 11 months ago

I have the same issue - the audio fragment duration is 1s longer than video:

$ mediastreamvalidator test-video/master.m3u8

--------------------------------------------------------------------------------
MUST fix issues
--------------------------------------------------------------------------------

Error: Different target durations detected
--> Detail:  Target duration: 4 vs Target duration: 3
--> Source:  audio-en-mp4a.40.2.m3u8
--> Compare: video-avc1-1.m3u8

--> Detail:  Target duration: 4 vs Target duration: 3
--> Source:  audio-en-mp4a.40.2.m3u8
--> Compare: video-avc1-2.m3u8

--> Detail:  Target duration: 3 vs Target duration: 4
--> Source:  video-avc1-3.m3u8
--> Compare: audio-en-mp4a.40.2.m3u8

Tried fragmenting audio both via ffmpeg and mp4fragmenter with the same result:

ffmpeg -y -i $VIDEO_IN \
       -vn \
       -c:a aac -b:a 128k -ac 2 -ar 48000 \
       -movflags +empty_moov+separate_moof -frag_duration 3000000 \
       -metadata:s:a:0 language=en \
       -f mp4 $VIDEO_OUT-aac.mp4

mp4fragment --fragment-duration 3000 $VIDEO_OUT-aac.mp4 $VIDEO_OUT-aacf.mp4

Packaging the same fMP4 into HLS/DASH using Shaka Packager produces no validator errors.