mifi / lossless-cut

The swiss army knife of lossless video/audio editing
GNU General Public License v2.0
24.01k stars 1.19k forks source link

Audio becomes "stretched" around cut from Smart cut only with H264+AAC MP4 files. #2050

Open helpimnotdrowning opened 1 week ago

helpimnotdrowning commented 1 week ago

The fewer issues I have to read, the more new features I will have time to implement, so I ask that you please try these things first

Steps to reproduce

  1. Download this Youtube stream https://www.youtube.com/watch?v=uIloWxQ3Rpo as VP9+OPUS with this command (1.14 gb file!):

    yt-dlp --format 303+251 --output "video-f%(format_id)s.%(ext)s" --merge-output-format mp4 https://www.youtube.com/watch?v=uIloWxQ3Rpo
  2. Download this Youtube stream https://www.youtube.com/watch?v=uIloWxQ3Rpo as H264+AAC with this command (1.55 gb file!):

    yt-dlp --format 299+140 --output "video-f%(format_id)s.%(ext)s" --merge-output-format mp4 https://www.youtube.com/watch?v=uIloWxQ3Rpo
  3. Copy this into an LLC project file (GH doesn't allow upload of LLC files):

    {
    version: 1,
    mediaFileName: 'dont-worry-about-this.mp4',
    cutSegments: [
    {
      start: 200.36682279880006,
      end: 479.1953581390453,
      name: 'SHINKIRO - Gura, Marine',
    },
    ],
    }

    Note the start time of the segment: it is in the middle of two keyframes for both videos: 303+251 image 299+140 image The difference between the 1st inner keyframe (3:24.004) and the start of the segment (3:20.367) is 3.64 seconds.

  4. Import 1st (303+251) video into LosslessCut

  5. Drag LLC file into LosslessCut, select "Load segments from the new file, but keep the current media"

  6. Export with the following settings:

Export mode Seperate files
Output container format mp4 - MP4 (MPEG-4 Part 14) (detected)
Input has 2 tracks Keeping 2 tracks
Output file name cut-${FILENAME}.${EXT}
Overwrite existing files True
Smart cut (experimental) True
Enable MOV Faststart True
Preserve all MP4/MOV metadata True
"ffmpeg" experimental flag False
  1. Repeat 4-6 with the other (299+140) file

With a sharp ear, you might be able to hear a slight difference in the beginning from the (correct) OPUS 303+251 version and the (bad) AAC 299+140 version of the song, around the 3.50~3.60 mark.

If not, you can listen to the difference file here, or create it manually in Audacity:

  1. Import both cut MP4s into Audacity (make sure ffmpeg is recognized by Audacity!) in this order: 299+140, 303+251

Notice the top 299+140 file is slightly "ahead" of the 303+251.

  1. Line up the unaffected audio portions:

    1. Right click on the label of the 2nd track and click "Swap Stereo Channels" (helps with alignment) image

    2. Pick some distinguishing segment, like where the audio suddenly spikes, Here, I use a spike occuring at 11.890~11.960 image

    3. Drag the 2nd track and line up the spikes as best as possible. Keep aligning and zooming in until the tracks are aligned at the sample level: Use smaller and smaller distinguishing features, like this spike and just bring the 2nd track as close as possible. Also note that these won't exactly match up since the 1st (AAC) track runs at 44.1kHz and the 2nd (OPUS) runs at 48kHz. image image image

    4. Unswap the 2nd track's stereo tracks with the same menu

    5. Select the entire 2nd track and Invert it (Effect/Special/Invert) image

    6. Listen to both tracks at the same time. The first "bump" should be almost entirely audible, then it quickly quiets down, afterwards only the resolution differences and (what I assume to be) the compression differences are audible.

In both the included difference file and (hopefully) any newly-created one, the only (real) audible bump should fade away sharply at the late 3.50-early 3:60 mark, which is "close enough" to the earlier difference of 3.64s between the segment start and next inner keyframe.

Expected behavior

Lossless Cut should not stretch the audio of the Smarty-ly cut segment on H264+AAC (299+140) MP4 files; it should result in (roughly) the same file as the correctly-manipulated VP9+OPUS (303+251) MP4 file (the audio is untouched, since it is never "keyframed" in the first place).****

Actual behavior

The audio for the H264+AAC (299+140) MP4 file is stretched around the boundary of the Smarty-ly cut segment. This does not happen for the VP9+OPUS MP4 (303+251) file.

Provide an error report

No error occurred.

{
  "err": false,
  "state": {
    "ffmpegExperimental": false,
    "preserveMovData": true,
    "movFastStart": true,
    "preserveMetadataOnMerge": false,
    "filePath": "C:\\Users\\helpi\\video-f303+251.mp4",
    "fileFormat": "mp4",
    "externalFilesMeta": {},
    "mainStreams": [
      {
        "index": 0,
        "codec_name": "vp9",
        "codec_long_name": "Google VP9",
        "profile": "Profile 0",
        "codec_type": "video",
        "codec_tag_string": "vp09",
        "codec_tag": "0x39307076",
        "width": 1920,
        "height": 1080,
        "coded_width": 1920,
        "coded_height": 1080,
        "closed_captions": 0,
        "film_grain": 0,
        "has_b_frames": 0,
        "sample_aspect_ratio": "1:1",
        "display_aspect_ratio": "16:9",
        "pix_fmt": "yuv420p",
        "level": -99,
        "color_range": "tv",
        "color_space": "bt709",
        "color_transfer": "bt709",
        "color_primaries": "bt709",
        "refs": 1,
        "id": "0x1",
        "r_frame_rate": "60000/1001",
        "avg_frame_rate": "31984200/533603",
        "time_base": "1/16000",
        "start_pts": 0,
        "start_time": "0.000000",
        "duration_ts": 42688240,
        "duration": "2668.015000",
        "bit_rate": "3381166",
        "nb_frames": "159921",
        "disposition": {
          "default": 1,
          "dub": 0,
          "original": 0,
          "comment": 0,
          "lyrics": 0,
          "karaoke": 0,
          "forced": 0,
          "hearing_impaired": 0,
          "visual_impaired": 0,
          "clean_effects": 0,
          "attached_pic": 0,
          "timed_thumbnails": 0,
          "captions": 0,
          "descriptions": 0,
          "metadata": 0,
          "dependent": 0,
          "still_image": 0
        },
        "tags": {
          "language": "eng",
          "handler_name": "VideoHandler",
          "vendor_id": "[0][0][0][0]"
        }
      },
      {
        "index": 1,
        "codec_name": "opus",
        "codec_long_name": "Opus (Opus Interactive Audio Codec)",
        "codec_type": "audio",
        "codec_tag_string": "Opus",
        "codec_tag": "0x7375704f",
        "sample_fmt": "fltp",
        "sample_rate": "48000",
        "channels": 2,
        "channel_layout": "stereo",
        "bits_per_sample": 0,
        "initial_padding": 312,
        "id": "0x2",
        "r_frame_rate": "0/0",
        "avg_frame_rate": "0/0",
        "time_base": "1/48000",
        "start_pts": 0,
        "start_time": "0.000000",
        "duration_ts": 128065968,
        "duration": "2668.041000",
        "bit_rate": "126184",
        "nb_frames": "133402",
        "extradata_size": 19,
        "disposition": {
          "default": 1,
          "dub": 0,
          "original": 0,
          "comment": 0,
          "lyrics": 0,
          "karaoke": 0,
          "forced": 0,
          "hearing_impaired": 0,
          "visual_impaired": 0,
          "clean_effects": 0,
          "attached_pic": 0,
          "timed_thumbnails": 0,
          "captions": 0,
          "descriptions": 0,
          "metadata": 0,
          "dependent": 0,
          "still_image": 0
        },
        "tags": {
          "language": "eng",
          "handler_name": "SoundHandler",
          "vendor_id": "[0][0][0][0]"
        }
      }
    ],
    "copyStreamIdsByFile": {
      "C:\\Users\\helpi\\video-f303+251.mp4": {
        "0": true,
        "1": true
      }
    },
    "cutSegments": [
      {
        "start": 200.36682279880006,
        "end": 479.1953581390453
      }
    ],
    "mainFileFormatData": {
      "filename": "C:\\Users\\helpi\\video-f303+251.mp4",
      "nb_streams": 2,
      "nb_programs": 0,
      "format_name": "mov,mp4,m4a,3gp,3g2,mj2",
      "format_long_name": "QuickTime / MOV",
      "start_time": "0.000000",
      "duration": "2668.041000",
      "size": "1173399590",
      "bit_rate": "3518385",
      "probe_score": 100,
      "tags": {
        "major_brand": "isom",
        "minor_version": "512",
        "compatible_brands": "isomiso2mp41",
        "encoder": "Lavf61.1.100"
      }
    },
    "rotation": 360,
    "shortestFlag": false,
    "effectiveExportMode": "separate",
    "outSegTemplate": "cut-${FILENAME}.${EXT}"
  },
  "platform": "win32",
  "version": "3.61.1"
}

Share the file

Appropriate files have been linked or pasted above.

Share log from developer tools

Exported log from dev console: -1719011931642.log