Audio not syncing?

[RenegadeEagle]

https://gist.github.com/RenegadeEagle/3752b204e5974c2ab65a So I have this code here. And I am attempting to combine all videos inside of my Videos folder, although it seems that the Audio is not Syncing correctly with all the videos. Is there a reason for this? Maybe a fix? Thanks.

[dnutcracker]

I've also witnessed the audio-out-of-sync problem - it happens when combining multiple short videos (2-3 seconds long) - the last segments start to lose audio sync (audio is lagging behind the video).

[bperin]

I have this same problem, any thoughts on a solution?

[sokolmkd]

Nothing so far?

[RazorPT]

Also noticed this happening after appending a third video, tried appending all three videos using the append example and then appending the first two and then the third video with merged video and still have this problem.

Problem happens with short videos but also with longer videos (8 seconds each)

Isn´t there a way to fix this?

[inbreaks]

Nothing so far?

[sannies]

Let me explain what happens here: The shortest video duration is ~40ms (1 out of ~25 frames per second), you can only have videos of the length 40,80,120,160 and so on millisecond. The shortest audio duration is 21 or 23ms (44.1 or 48KHz).

You can most likely already imagine what is happening: Even if you try your best you might run into a situation where audio and video have different durations up to (audio frame duration / 2) = 10.5 or 11.5ms.

If this is added up -> ouch! You need to compensate for that if you append multiple videos and add an extra audio frame or omit an audio frame.

Sometimes i've seen that encoders systematically produce shorter audio than video or the other way around. If you append that -> ouch, offset! If the video is longer than the audio you cannot use the part without audio when appending as this will lead to an offset/lip-sync issue in the next part. You might try to compensate by adding silence.

Does that help to understand what's going on?

Grüße, Sebastian

terry notifications@github.com schrieb am Di., 25. Apr. 2017 um 09:51 Uhr:

Nothing so far?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/sannies/mp4parser/issues/21#issuecomment-296948030, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKUD33kl103MocbsM8fR4Tk8GEfZLbRks5rzaXugaJpZM4C0pgO .

[buntupana]

This functions will append videos with no out of sync problems

`@Throws(Exception::class)
 fun appendVideos(videoPathList: List<String>, targetFilePath: String) {

    val movies = videoPathList.flatMap { file -> listOf(MovieCreator.build(file)) }

    val finalMovie = Movie()

    val videoTracksTotal = mutableListOf<Track>()
    val audioTracksTotal = mutableListOf<Track>()

    var audioDuration = 0.0
    var videoDuration = 0.0

    movies.forEach { movie ->

        val videoTracks = mutableListOf<Track>()
        val audioTracks = mutableListOf<Track>()

        movie.tracks.forEach { track ->

            val trackDuration = track.sampleDurations.toList()
                .map { t -> t.toDouble() / track.trackMetaData.timescale }.sum()

            if (track.handler == "vide") {
                videoDuration += trackDuration
                videoTracks.add(track)
            } else if (track.handler == "soun") {
                audioDuration += trackDuration
                audioTracks.add(track)
            }
        }

        // Adjusting Durations
        adjustDurations(videoTracks, audioTracks, videoDuration, audioDuration).let {
            audioDuration = it.audioDuration
            videoDuration = it.videoDuration
        }

        videoTracksTotal.addAll(videoTracks)
        audioTracksTotal.addAll(audioTracks)
    }

    if (videoTracksTotal.isNotEmpty() && audioTracksTotal.isNotEmpty()) {
        finalMovie.addTrack(AppendTrack(*videoTracksTotal.toTypedArray()))
        finalMovie.addTrack(AppendTrack(*audioTracksTotal.toTypedArray()))
    }

    val container = DefaultMp4Builder().build(finalMovie)

    val fos = FileOutputStream(targetFilePath)
    val bb = Channels.newChannel(fos)
    container.writeContainer(bb)
    fos.close()
}

class Durations(val audioDuration: Double, val videoDuration: Double)

private fun adjustDurations(
    videoTracks: MutableList<Track>,
    audioTracks: MutableList<Track>,
    videoDuration: Double,
    audioDuration: Double
): Durations {

    var diff = audioDuration - videoDuration
    val tracks: MutableList<Track>
    var durationOperator: Double
    val isAudioProblem: Boolean

    when {
        // audio and video match, no operations to perform
        diff == 0.0 -> {
            return Durations(audioDuration, videoDuration)
        }
        // audio tracks are longer than video
        diff > 0 -> {
            tracks = audioTracks
            durationOperator = audioDuration
            isAudioProblem = true
        }
        // video tracks are longer than audio
        else -> {
            tracks = videoTracks
            durationOperator = videoDuration
            diff *= -1.0
            isAudioProblem = false
        }
    }

    // Getting the last track in order to operate with it
    var track: Track = tracks.last()
    var counter: Long = 0

    // Reversing SampleDuration list
    track.sampleDurations.toList().asReversed().forEach { sampleDuration ->

        // Calculating how much this track need to be re-adjusted
        if (sampleDuration.toDouble() / track.trackMetaData.timescale > diff) {
            return@forEach
        }
        diff -= sampleDuration.toDouble() / track.trackMetaData.timescale
        durationOperator -= sampleDuration.toDouble() / track.trackMetaData.timescale
        counter++
    }

    if (counter != 0L) {
        // Cropping track
        track = CroppedTrack(track, 0, track.samples.size - counter)

        //update the original reference
        tracks.removeAt(tracks.lastIndex)
        tracks.add(track)
    }

    // Returning durations
    return if (isAudioProblem) {
        Durations(durationOperator, videoDuration)
    } else {
        Durations(audioDuration, durationOperator)
    }
}`