High disk usage compared to other scripts

[AKL55]

Was testing the script with default values and i noticed that it needs to read the whole file in order do copy/encode, this makes it extremely slow for large files.

Some tests on 7.25GB file Using -c copy 12s cut took 45s on normal hard drive

Same cut on encode.lua & webm.lua makes the cut instant without reading the whole file. I'm on Windows 10

[familyfriendlymikey]

Thanks for reporting! I didn't notice this because I pretty much only use the list action. Seems like it's because of the -map 0 flag. There's always something with ffmpeg...

If you want to use it right now you can just add this to your config.lua:

ACTIONS.COPY = function(d)
    local args = {
        "ffmpeg",
        "-nostdin", "-y",
        "-loglevel", "error",
        "-ss", d.start_time,
        "-t", d.duration,
        "-i", d.inpath,
        "-pix_fmt", "yuv420p",
        "-c", "copy",
        "-avoid_negative_ts", "make_zero",
        utils.join_path(d.indir, "COPY_" .. d.channel .. "_" .. d.infile_noext .. "_FROM_" .. d.start_time_hms .. "_TO_" .. d.end_time_hms .. d.ext)
    }
    mp.command_native_async({
        name = "subprocess",
        args = args,
        playback_only = false,
    }, function() print("Done") end)
end

[AKL55]

Fixed! Thanks.

[familyfriendlymikey]

Will try to merge a solution to main later

[AKL55]

Will try to merge a solution to main later

I forgot to mention that this also happens when using the encoding option; but, it worked for the copy command

[familyfriendlymikey]

The encode action is a different situation, I figure that if someone is using the encode action it's because they want a very precise cut, otherwise the copy action is better. Because of that ffmpeg has to process the entire file up to the point you want to cut. If you're okay with a little less precision you can try:

ACTIONS.ENCODE = function(d)
    local args = {
        "ffmpeg",
        "-nostdin", "-y",
        "-loglevel", "error",
        "-ss", d.start_time,
        "-t", d.duration,
        "-i", d.inpath,
        "-pix_fmt", "yuv420p",
        "-crf", "16",
        "-preset", "superfast",
        utils.join_path(d.indir, "ENCODE_" .. d.channel .. "_" .. d.infile_noext .. "_FROM_" .. d.start_time_hms .. "_TO_" .. d.end_time_hms .. d.ext)
    }
    mp.command_native_async({
        name = "subprocess",
        args = args,
        playback_only = false,
    }, function() print("Done") end)
end

[familyfriendlymikey]

Let me know if the encode thing I sent fixes your issue.

[AKL55]

The encode action is a different situation, I figure that if someone is using the encode action it's because they want a very precise cut, otherwise the copy action is better. Because of that ffmpeg has to process the entire file up to the point you want to cut. If you're okay with a little less precision you can try:

ACTIONS.ENCODE = function(d)
  local args = {
      "ffmpeg",
      "-nostdin", "-y",
      "-loglevel", "error",
      "-ss", d.start_time,
      "-t", d.duration,
      "-i", d.inpath,
      "-pix_fmt", "yuv420p",
      "-crf", "16",
      "-preset", "superfast",
      utils.join_path(d.indir, "ENCODE_" .. d.channel .. "_" .. d.infile_noext .. "_FROM_" .. d.start_time_hms .. "_TO_" .. d.end_time_hms .. d.ext)
  }
  mp.command_native_async({
      name = "subprocess",
      args = args,
      playback_only = false,
  }, function() print("Done") end)
end

Weird, The updated code made the timestamp buggy, it had a fake lenght, but the cut was precise old code had the same precision without the buggy timestamp

The old code took 1Minute to encode a 4s cut

webm.lua somehow took 4s, and also had the same precision.

[AKL55]

The issue for slow encoding was subtitles, adding -sn or -map 0 + -c:s copy fixed the issue. Video files with ass and srt subs was the problem to be more specific.

[familyfriendlymikey]

Thank you very much for reporting back!

Would you be able to point me to a video file where I can test this? If it's not convenient for you then no worries.

Also for posterity, the reason why the COPY action was slow was because the -map 0 option was mapping the "data" stream which for some reason made it very slow. So the default copy action now uses -map 0 -dn to exclude the data stream.

[AKL55]

My ffmpeg default commands

ACTIONS.COPY = function(d)
    local args = {
        "ffmpeg",
        "-nostdin", "-y",
        "-loglevel", "error",
        "-ss", d.start_time,
        "-t", d.duration,
        "-i", d.inpath,
        "-c", "copy",
        "-map", "0",
        "-dn",
        "-c:s", "copy",
        "-avoid_negative_ts", "make_zero",

 ACTIONS.ENCODE = function(d)
    local args = {
        "ffmpeg",
        "-nostdin", "-y",
        "-loglevel", "error",
        "-ss", d.start_time,
        "-t", d.duration,
        "-i", d.inpath,
        "-c:a", "aac", "-b:a", "200K",
        "-c:v", "libx264", "-crf", "16", "-preset", "superfast",
        "-pix_fmt", "yuv420p",
        "-map", "0",
        "-c:s", "copy",

I did trim the files. you can try copying the files and merging with losslesscut to increase the filesize

File 1 = 1 ASS sub + MKV cover art COPY and Encode gives a 0 byte file with "-map", "0". https://gofile.io/d/KEw2c0

File 2 = 1 SRT Sub + MKV cover art Encode = Need to remove "-map", "0", or encoding takes too long on the original file 5gb Copy = Need to remove "-map", "0", or else gives 0 byte file I think the problem was the embed cover art https://gofile.io/d/Bbspcw

File 3 = Multiple PGS subs, also tested with removing others subs and only leaving the file with 1 pgs sub Works fine with default values, Both encoding & Copy https://gofile.io/d/frPQxI

File 4 = 2 PGS and 2 ASS subs COPY Works fine Encoding = Only worked well if using -sn, if not, video timestamp is incorrect https://gofile.io/d/CIdzZI

if using -c:s copy video encoding is fast but a lot of times we get negative timestamp, if not using -c:s copy or specifying any sub codec, video encoding takes too long for no reason

Seems odd the way that, if you Cut using COPY, the subs are fine but, if you reencode the video keeping subs the video timestamps is incorrect.

I will try to see how to encode the video with subs without incorrect timestamps later.

[familyfriendlymikey]

Thanks a ton, this is really helpful. Will check it out later!