Question about audio sync and two-pass libx264 plus ac3 output with ffmpeg

[loshlee]

I'm playing with a formula for a two-pass ffmpeg encoding command to use for my Mac (Silicon - M1), and I thought I might ask for advice here. There's a chance that the method used for detelecine in other-video-transcoding may be a little more robust than the HandBrake-cli's detelecine option for my sources. I have a capture device for over the air (ATSC?) HD signals, and it delivers only telecined 29.97 fps mpegts content. The detelecine option for the two-pass-transcode.rb works very well for me to create 24 fps output for my Apple TV, but from my testing, ffmpeg's fieldmatch filter chain may not leave quite as many duplicate frames. There are always a few, because of less than perfect input from transmission, but think it might be worthwhile for the sake of quality to try to refine the ffmpeg version. I need to stay with libx264 right now, because Mac Home Sharing using Apple TV doesn't appear to successfully handle libx265 encoding yet (all my efforts have so far displayed a black window, although other players have no trouble with them).

That's the background. The issue is that when I run my transcoding command, the audio turns out about 4 minutes less in duration than the original. The two-pass-transcode.rb doesn't appear to create that problem. I've devised a method to work around this by using an ffmpeg pipe to sox for reducing the tempo of the ac3 audio component with a formula based on original tempo divided by the sum of original duration and the observed desired increase in duration, (the desired audio duration currently being greater) then applying the result of that calculation to the original via sox's change tempo command. Changing tempo by a factor less than one increases duration, changing by a factor greater than one decreases duration. (Sox actually displays a rough textual representation of a VU meter in the command line during the process - it's really pretty impressive. I'll share my AppleScript wrapper for the shell command after I've refined it as much as my capability will allow.)

Anyway, I was hoping you might take a look at my transcode command and tell me if you can identify whatever might be causing the A/V sync problem (shorter duration for audio by about 4 seconds per hour). This is just a single pass routine. The reduced duration for output also occurs if I copy audio without encoding. There is one video stream and one for audio.

ffmpeg -r 29.97 -i input.ts -map 0:0 -c:v:0 libx264 -b:v 4800k -fps_mode cfr -vf fieldmatch=order=tff:combmatch=none,decimate,crop=1920:800:0:140 -map 0:1 -c:a ac3 -b:a 640k -metadata:s:a:0 title\= -disposition:a:0 default -sn -metadata:g title\= -default_mode passthrough -movflags +faststart output.mp4

Thanks.

[loshlee]

If I use hevc_videotoolbox -pix_fmt:v p010le instead of libx264 and copy the audio rather than encode it, the audio is about 3 seconds shorter than the video per hour rather than 4 seconds shorter.

[loshlee]

I discovered how to remedy the audio sync issues. Here's the command that works for my video input, and I'm including it in the two-pass version. It doesn't contain a method for dealing with the temporary file generated by the two-pass call, but that is trivial. I still have two questions, because the output still contains infrequent duplicate frames. The questions are:

1. Is there really any benefit for using the two-pass method for this ffmpeg encoding process?
2. Is there any way to remove the occasional duplicate frame in the output? Apparently, the source video occasionally contains (as infrequently as once every 10 or 15 minutes) a single progressive frame where it should contain 2 interlaced frames in the telecine pattern. That means that rarely, there is a non-telecine group of 4 frames where there should be 5. Is there any way to amend the command so that it can prevent the duplicate frame for those rare instances?

ffmpeg -i input.ts -map 0:0 -filter:v fieldmatch=order=tff:combmatch=none,decimate,crop=1920:800:0:140 -pass 1 -c:v h264_videotoolbox -b:v 4000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v "title=" -disposition:v default -map 0:1 -c:a:0 copy -metadata:s:a:0 "title=" -disposition:a:0 default -sn -metadata:g "title=" -movflags disable_chpl -an -f null /dev/null && ffmpeg -i input.ts -map 0:0 -filter:v fieldmatch=order=tff:combmatch=none,decimate,crop=1920:800:0:140 -pass 2 -c:v h264_videotoolbox -b:v 4000k -color_primaries:v bt709 -color_trc:v bt709 -colorspace:v bt709 -metadata:s:v "title=" -disposition:v default -map 0:1 -c:a:0 copy -metadata:s:a:0 "title=" -disposition:a:0 default -sn -metadata:g "title=" -movflags disable_chpl output.mp4

[loshlee]

I've reached the conclusion that the two-pass method is a waste of time. I have two types of digital film sources to transcode. The first appears to be 3:2 pulldown and the second appears to be 4:1 where every 5th frame is an exact duplicate of every 4th. I find that "fieldmatch=order=tff:combmatch=none,decimate" works well for the former and "decimate=cycle=5" works well for the latter. That's purely an observation. For either filter chain, output must have a 24 fps rate. I continue to experiment.

[lisamelton]

@loshlee I'm sorry you're having this problem. And, my apologies, but I missed all your previous comments on this issue. Something appears to be wrong with my GitHub notifications.

Doing a two-pass transcode with ffmpeg is more complicated than HandBrakeCLI. That said, the ratecontrol system, single- or two-pass, is completely orthogonal to the issues you're having.

[loshlee]

I meant to say that the two-pass method was a waste of time using ffmpeg. Two-pass-transcode.rb is not a waste of time by any means. Thanks again.