search

Vanilagy / mp4-muxer

MP4 multiplexer in pure TypeScript with support for WebCodecs API, video & audio.
https://vanilagy.github.io/mp4-muxer/demo
MIT License
419 stars 32 forks source link

audio video way out of sync in latest version #49

Closed RavikumarTulugu closed 4 months ago

RavikumarTulugu commented 4 months ago

I am trying to understand and debug what is happening, may be it is the issue with our application. please bear with me. All good except the audio is very fast and way out of sync with the video, the audio randomly plays smooth at few intervals , rest it goes pretty fast and unintelligible. I recorded a 10min youtube video, and found that the audio duration is only 1 minute while the video is full 10 mins. The video shows up fine. I am pretty sure, i saw this working few months back. here is the output of mediainfo command of the recorded mp4 video.

mediainfo 'soundkone.net_recording_2024-06-03 10_53_44.mp4'

General Complete name : soundkone.net_recording_2024-06-03 10_53_44.mp4 Format : MPEG-4 Format profile : Base Media Codec ID : isom (isom/avc1/mp41) File size : 32.7 MiB Duration : 10 min 6 s Overall bit rate : 451 kb/s Frame rate : 5.826 FPS Encoded date : 2024-06-03 05:23:47 UTC Tagged date : 2024-06-03 05:23:47 UTC

Video ID : 1 Format : AVC Format/Info : Advanced Video Codec Format profile : Constrained Baseline@L3.1 Format settings : 1 Ref Frames Format settings, CABAC : No Format settings, Reference frames : 1 frame Format settings, Slice count : 2 slices per frame Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 10 min 6 s Source duration : 10 min 7 s Bit rate : 435 kb/s Width : 800 pixels Height : 600 pixels Display aspect ratio : 4:3 Frame rate mode : Variable Frame rate : 5.826 FPS Minimum frame rate : 1.562 FPS Maximum frame rate : 12.508 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.156 Stream size : 31.5 MiB (96%) Source stream size : 31.5 MiB (96%) Title : mp4-muxer-hdlr Encoded date : 2024-06-03 05:23:47 UTC Tagged date : 2024-06-03 05:23:47 UTC Color range : Limited Color primaries : BT.601 NTSC Transfer characteristics : BT.601 Matrix coefficients : BT.601 mdhd_Duration : 606960 Codec configuration box : avcC

Audio ID : 2 Format : AAC LC Format/Info : Advanced Audio Codec Low Complexity Codec ID : mp4a-40-2 Duration : 1 min 10 s Source duration : 1 min 10 s Bit rate mode : Constant Bit rate : 132 kb/s Channel(s) : 1 channel Channel layout : M Sampling rate : 48.0 kHz Frame rate : 46.875 FPS (1024 SPF) Compression mode : Lossy Stream size : 1.11 MiB (3%) Source stream size : 1.11 MiB (3%) Title : mp4-muxer-hdlr Encoded date : 2024-06-03 05:23:47 UTC Tagged date : 2024-06-03 05:23:47 UTC mdhd_Duration : 70187

RavikumarTulugu commented 4 months ago

a little better after changing the codec to opus and the 'faststart' parameter is set to 'fragmented' but still the duration is shorter and 90% of the audio is missing from the resulting file. media info command output pasted below.

mediainfo 'soundkone.net_recording_2024-06-04 19_57_03.mp4' General Complete name : soundkone.net_recording_2024-06-04 19_57_03.mp4 Format : MPEG-4 Format profile : Base Media Codec ID : iso5 (iso5/iso6/mp41) File size : 15.1 MiB Duration : 15 min 48 s Overall bit rate : 134 kb/s Frame rate : 4.338 FPS Encoded date : 2024-06-04 14:27:05 UTC Tagged date : 2024-06-04 14:27:05 UTC

Video ID : 1 Format : AVC Format/Info : Advanced Video Codec Format profile : Constrained Baseline@L3.1 Format settings : 1 Ref Frames Format settings, CABAC : No Format settings, Reference frames : 1 frame Format settings, Slice count : 2 slices per frame Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 15 min 48 s Bit rate : 112 kb/s Width : 800 pixels Height : 600 pixels Display aspect ratio : 4:3 Frame rate : 4.338 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.054 Stream size : 12.6 MiB (83%) Title : mp4-muxer-hdlr Encoded date : 2024-06-04 14:27:05 UTC Tagged date : 2024-06-04 14:27:05 UTC Color range : Limited Color primaries : BT.601 NTSC Transfer characteristics : BT.601 Matrix coefficients : BT.601 Codec configuration box : avcC

Audio ID : 2 Format : Opus Codec ID : Opus Duration : 2 min 38 s Bit rate : 128 kb/s Sampling rate : 48.0 kHz Compression mode : Lossy Stream size : 2.42 MiB (16%) Title : mp4-muxer-hdlr Encoded date : 2024-06-04 14:27:05 UTC Tagged date : 2024-06-04 14:27:05 UTC

RavikumarTulugu commented 4 months ago

few more observations, when the video track is omitted and just audio track is recorded from the youtube video, the duration and the quality of the audio are recorded fine in the resulting mp4 file.

Vanilagy commented 4 months ago

This sounds like an issue in your application. You should log out the timestamps for both of your media tracks and see where it's going wrong.

There might be valuable information for you in this issue here: https://github.com/Vanilagy/mp4-muxer/issues/17