mifi / lossless-cut

The swiss army knife of lossless video/audio editing
GNU General Public License v2.0
24.74k stars 1.22k forks source link

Lossless cut Time Code counter drifts out. #1470

Open Doug183 opened 1 year ago

Doug183 commented 1 year ago

I have a lot of issues to go through, so in order to make it easier for me to help you, I ask that you please try these things first

Operating System

MacOS 12

Steps to reproduce

The time code counter in Lossless Cut drifts "long" for long media files - at least ones that are in QuickTime or H264, 23.976 fps. My test file is ~49 minutes and the lossless cut counter at the end of the file seems to be ~3 seconds “longer” that the file length. I have two methods of observing this behavior, described in tests 1,2,3 below.

***FYI, this problem is reproducible with other long media files.

My sources files which can be downloaded to test: Two files, created using a film scanner called a ScanStation. 1 - Quicktime ProResHQ (1920x1080). 23.976 fps. Audio PCM. DOWNLOAD smaller 4.3GB File link (480x270) for testing. 2 - Mp4 - 960x540 (960x540) DOWNLOAD .mp4 1.4GB. 23.976 fps. AAC

Tests. 1) Using Adobe Premiere - There seems to be no timecode drift: a) If I stack Original files 3 files in a timeline, they perfectly line up and stay in sync.
b) Also the Time Code visually burned in on the .mp4 file matches the Timecode of the timeline in Adobe premiere. c) I can create the timeline 4 different ways in Premiere (1 generated from each clip or create a completely new timeline and all media files in each timeline all consistently line up.)

2) Using Quicktime 7 - in a virtual Machine recently, Quicktime 7 reports accurate timecode. This method I no longer use.

3) Compressor 4.6.3 - reports the time code accurately of 49min 12seconds 22 Frames

3) Lossless Cut a) Reports the file is: 49 minutes, 15 seconds, and 21 frames. - (~ 3 seconds longer)

4) Using QuickTime 10.5 - Counter reports 49 minutes and 16 seconds. (~ 3 seconds longer)

5) VLC - Counter reports 49 minutes and 16 seconds. (~ 3 seconds longer)

More Tests: 1) Exporting from Lossless Cut: a) Exporting 1920x1080 ProResHQ file, results in what seems to be a perfect duplicate with duplicate results.
a1) If I put this file back into Adobe Premiere, it will cut perfectly back into the timeline. Frame accurate. a2) Adobe Premiers reports the media length as being 49 min 12 seconds 22 Frames. a3) QuickTime X reports the file being 49 minutes and 16 seconds. a4) Lossless Cut reports the file being 49 minutes and 16 seconds.

b) Exporting 960x540 H264 file, results in what seems to be another perfect duplicate with duplicate results.
b1) If I put this file back into Adobe Premiere, it will cut perfectly back into the timeline. Frame accurate. b2) Adobe Premiers reports the media length as being 49 min 12 seconds 22 Frames. b3) QuickTime X reports the file being 49 minutes and 16 seconds. b4) Lossless Cut reports the file being 49 minutes and 16 seconds.

So the issues seems only be limited to how the math is being handled somewhere internally in Lossless Cut. Although not directly related, interesting that Quicktime X and VLC have the same issue as Lossless Cut.

Expected behavior

Adobe Premiere Screen shots. Shot1_Adobe Premiere Shot2_Adobe Premiere

Lossless Cut Screen Shots Shot3_Lossless_Start Shot4_Losless End

Compressor Shot5_Compressor

Actual behavior

Can this be fixed?

Share log

No response

mifi commented 1 year ago

thanks for your detailed report. Unfortunately the file you shared takes forever to download (not sure why).

It definitely seems to be a problem with these particular files, if it happens in other popular players like quicktime. Does it report the wrong duration if you open the video in Chrome or Safari browser too?

Some things to try:

What happens if you seek to the last 2 seconds of the video in losslesscut and start playing? does it playback the end of the video, or something else happens?

What happens if you set the END cutpoint to say 1 second before the end of the video, and then export. Then if you open the output file in premiere, is the video cut-off by a few seconds or is it the full original length?

Doug183 commented 1 year ago

Thanks I will do those tests.

I also will post some smaller versions of those files so you can test yourself.

The good news is now I remember that a few yeas ago, I had a great friend/ human/programmer do a cool project with ffmpeg and ffprobe (replace the last 6 seconds of hundreds of videos and HLS files) and in that project we discovered something like... although ffprobe will give the frame rate 18fps, 24fps, 23.976fps, 29.997 etc... and will give the the length of video file in ... microseconds? (I forget). There is some internal Math problem that it does wrong. My friend figured out away around it (keep everything in microseconds?) and then convert back.? Ok enough of my half memories, I will just get him to post.

Also thank you for this amazing project you are doing. With the death of Quicktime 7, its is amazing. FYI, I bought on the App Store as well and will be happy to donate even more, if we can get this fixe, as it will save me 50-60 hours over the course of using your app.

Doug

Doug183 commented 1 year ago

Ok, here is the snippet of code from a python script Andy Fuhr created to calculate the correct frame count using ffpobe and ffmpeg. He has given me permission to post. I hope its complete enough and works with your code to address the time codes counter being off. Let me know what you think.

# FFPROBE GIVES Frame RATE like:
self.frame_rate = "24000/1001"
self.frame_rate = "2997/125"
self.frame_rate = "18/1"

# TO GET THE RIGHT  TIMECODE NUMBER WHEN DURATION IS IN fractional SECONDS

# Convert Frame rate
xlist = self.frame_rate.split("/")
converted_frame_rate = float(xlist[0]) / float(xlist[1])
converted_duration = self.convert(int(round(float(self.duration))))

def convert(self, seconds):

       seconds = seconds % (24 * 3600)
       hour = seconds // 3600
       seconds %= 3600
       minutes = seconds // 60
       seconds %= 60
       if hour <= 0:
           return "%02d:%02d" % (minutes, seconds)
       else:
           return "%d:%02d:%02d" % (hour, minutes, seconds)
Dean-Corso commented 1 year ago

Hi guys,

I also found a issue with the time & frame handling. I see its not hitting / showing always the right frame on specific time. Just found that problem after using "detect scene changes" and checking the single created segments whether they really match with a scene change and there the SC is start is set one frame before (old scene). Maybe a problem with ffmpeg itself. Example: I made some images.... I1_2023-02-20_195820 ...in this first image above I do click on the 37. segment and it does show the wrong frame what is the last frame of segment 36. Now lets have a look at the same time code in VirtualDub... I2_2023-02-20_195820 ...what is showing the right frame / start of scene segment 37. If I now seek one frame forward in LC then its showing the right frame start of scene 37 but with other timecode over segment bar.... I3_2023-02-20_195820 ...so that means that not always the right frame is showing on the right time code what is problematic on cutting or handling segments. I hope that you can find and fix that wrong / missmatch frame issues.

Request: Is it possible that you can make a liquid showing frame change if the user does move the pointer / slider in the bar? So I always have to wait if I move this slider for a while. Just wanna know whether you can update those frame changes also in realtime or not. Or do you just make each time a new image using ffmpeg?

A display about the frames (value of is frame like in VD) would be also nice with a direct frame jumper (enter frame value / jump there).

Dean-Corso commented 1 year ago

Found another problem during preview of silent scenes. So they do preview some sound instead of silence. Look..audio file with some silences I made.... R2_2023-02-20_224923 ...now LC does find them... R1_2023-02-20_224923 ...but if I preview / loop the single silence segments then it does output some sound but should output the no sound part. Must be again some little missmatch so that the preview does not match exactly but the time codes are fine. Just as info.

mifi commented 1 year ago

@Dean-Corso I think your problem is different from OP. OP was about timecode drifting by many seconds over a long file. Your issue seems to be about inaccuracies (one frame or so) of the seeking. I think your issue is more related to this: #1487

mifi commented 2 months ago

Ok, here is the snippet of code from a python script Andy Fuhr created to calculate the correct frame count using ffpobe and ffmpeg. He has given me permission to post. I hope its complete enough and works with your code to address the time codes counter being off. Let me know what you think.

losslesscut calculates FPS in a similar way https://github.com/mifi/lossless-cut/blob/5fdf86feacf9722abe21f0b81c4473861912f4bc/src/renderer/src/ffmpeg.ts#L581

However, in order to format the duration e.g. 00:00:01.123, the FPS isn't actually used at all, because ffprobe returns the time value, so FPS is not needed to calculate times.

This is also evident from your sample python code, because the calculated variable converted_frame_rate is not being used at all in that code.