Problems with duplicate frames when encoding files of greater duration

[loshlee]

When I asked for help with my transition to a new M1 MacBook Pro, the question arose as to why I still used transcode-video at all. The question that I'm asking here may provide a partial explanation for that, but is rather tedious to state, so please bear with me.

Using the current version of other-transcode, I'm transcoding progressive, telecine 60 fps video to progressive 23.976 (24000/1001) using the --mp4 and --detelecine options. I use .mp4 because I'm using Apple Home Sharing to view the videos and so far I haven't been able to get the Mac's TV.app to accept .mkv files. I'm getting occasional duplicate frames in the transcoded output. I detect the frames visually (because I've gotten used to seeing them), but I confirm that they are indeed duplicate by viewing with IINA for Mac, which provides stepping through frames with "," and "." by default. I've observed that I get different results when I perform the operation on a file of longer duration from those I get when performing the same operation on files of shorter duration that contain passages in which duplicate frames occur. The difference is that encoding the shorter duration files results in no duplicate frames. This and the fact that Handbrake invoked using transcode-video results in no duplicate frames for the same operation convince me that there's no problem with the source video.

To summarize, it appears that, in my case, Handbrake can avoid duplicate frames better than ffmpeg for the other-transcode toolbox operation. Handbrake takes a lot longer, of course.

I can provide some evidence of my observation through video examples, but it's not practical to try to share a 2-hour file of 1080p video, of course. So I have two questions. Before asking those, here's an example of the command line I use:

other-transcode --detelecine --mp4 --crop auto --target 3600 'path/to/prog_tel_60_fps_file.ts'

I can, if needed, provide both logs, the log for the operation on the larger file producing duplicate frames, and the log for the operation on the brief excerpt that produces no duplicates. BTW, I've already tried creating the excerpt with nearly the same duration from the original large file, and the duplicates are still produced in the same places.

What can I try adding to the above command line to prevent (seemingly random) duplicate frames for the command, and if there isn't any such method, is there a another command I can try with ffmpeg or something that might remove the duplicates in a separate operation without changing quality or anything else?

There are many reasons I hope to find a compatible workflow using other-transcode, but the Handbrake method does at present provide a viable solution, albeit a much slower one.

Thanks.

[lisamelton]

@loshlee I'm sorry that you're having this problem. Where are you getting this 60 FPS video? Because normally only 29.97 FPS video is telecined. And progressive video is, by definition, never telecined. So, I'm confused.

The --detelecine option in other-transcode is implemented very differently from the --detelecine option in HandBrakeCLI. My version uses a simple fieldmatch=order=tff:combmatch=none,decimate filter set in ffmpeg. The decimate filter's purpose is to drop duplicate frames. However, it doesn't work all that well if the source frame rate is so high.

Rather than using the --detelecine option, why don't you try --rate 24000/1001 instead?

I hope that helps.

[loshlee]

It’s not a very standard source, I guess. It’s a set top box/DVR combination from which I import videos for Home Sharing so that the content can be viewed on TV's as well as mobile devices and computers. You may remember getting an email or tweet or something a few years ago containing information that a software update for the iPad Pro with the TV app had provided the ability to decode 5.1 ac3 audio so that including the traditional audio file pair for Home Sharing might no longer be necessary for some. That was me.

I think it might be reasonable and possible to provide the short examples (about 5 seconds) of the files I’m working with (one transcoded from the full duration source with the dup present, another transcoded from the excerpt which has the same duration as the clip, and another which is the source of the shorter clip - un-transcoded). I can even provide the estimated frame numbers displayed by IINA info, specifying the dups' frame numbers so you can observe the double-tap necessary to proceed past them when stepping through (as well as to alternately press "," and "." without seeing any video motion between the two frames). The video transcoded this way is perfectly smooth and clear except for the duplicated frame.

I’ve definitely tried 24000/1001 and detelecine both, separately and together, even substituting 23.976 for 24000/1001 on several tries. That results in 1 of 4 frames being duplicates, and is quite harsh looking. I can provide a clip of the result of that transcode method so you can observe the uneven flow, if you like. I’ve tried the ffmpeg command line provided in the log directly in Terminal, and get the same result. The only thing that seems to make any difference is the duration of the video segment being encoded. I haven’t really determined the maximum length of a passage that can be transcoded without introducing the spurious duplicate frames, but I have considered trying to split the video, transcode, then re-concatinate, which may be possible. I just doubt that the segments will rejoin with smooth seams. I haven’t ruled it out yet, though.

Thanks for responding.

[loshlee]

I'm closing this for now. Tried the same routine on a similar source using a new version of ffmpeg, and could find no instance of this problem. Perhaps a problem was solved through ffmpeg development which had the side effect of solving this one. Thanks again for other_video_transcoding.

[loshlee]

I think I've found a better workaround using video-transcoding. I'm using --mp4, but Is there an other-transcode equivalent for the command you provided at the link below? Or something close, perhaps? Thanks. MKV to MKV, reducing size but leaving interlaced?

[lisamelton]

@loshlee I don't understand what you mean. Which command? There are a lot of them in that issue.

[loshlee]

Thanks. Sorry, too excited, I guess. It goes like this. transcode-video --mp4 --target big --filter no-detelecine --filter no-decomb --filter no-comb-detect --filter no-deinterlace --force-rate 29.970 -E bff=1 input.ts -o output.mp4

[lisamelton]

@loshlee Ah. OK, you can add --yadif-params bff=1 to your command line to do that.

Does that answer your question?

[loshlee]

I think so. Are the rest of the options covered in other-transcode help? It's not critical, I can just plug the log output into an AppleScript. Did you pay any attention to my AppleScript droplet source for ffmpeg-normalize that I announced on the ffmpeg-users list yesterday? I'll put it into something like that. I have plenty of them. A lot. Thanks again.

[lisamelton]

@loshlee The options in other-transcode do not always correspond to the HandBrake filters available from transcode-video. To see all the available options, use --help full.

I don't follow that mailing list so I didn't see it.

You're welcome, I'll close this again.