ekisu / mpv-webm

Simple WebM maker for mpv, with no external dependencies.
MIT License
586 stars 33 forks source link

Source FPS is not preserved (CFR → VFR) #131

Open Igetin opened 3 years ago

Igetin commented 3 years ago

mpv version and platform

Operating system: Fedora 34
This mpv was installed from the official Fedora RPM repository via dnf.

mpv 0.33.1 Copyright © 2000-2020 mpv/MPlayer/mplayer2 projects
 built on UNKNOWN
FFmpeg library versions:
   libavutil       56.70.100
   libavcodec      58.134.100
   libavformat     58.76.100
   libswscale      5.9.100
   libavfilter     7.110.100
   libswresample   3.9.100
FFmpeg version: 4.4

Description

mpv-webm will not preserve the exact framerate when the source file has a constant framerate, and the output file will have a variable framerate.

Steps to reproduce

  1. Open a video file (CRF) with mpv
  2. Press Shift+W to bring up the menu
  3. Go to the options page and set output format to "MP4 (h264/AAC)", and press enter
  4. Set the start and end time
  5. Encode
  6. The resulting file will have a slightly different framerate than the source file

The script was run as-is. No custom settings were specified in script-opts/webm.conf.

Details

Source file ffprobe example:

$ ffprobe puipui.mp4 |& rg Video: --after-context=3
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 3219 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 47.95 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]

Output file ffprobe example:

$ ffprobe puipui-\[01.18.161-01.24.167\]-audio.mp4 |& rg Video: --after-context=3
  Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709/unknown/unknown), 1920x1080 [SAR 1:1 DAR 16:9], 3336 kb/s, 24.14 fps, 23.98 tbr, 24k tbn, 48k tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]

In this case, the FPS changed from 23.98 FPS to 24.14 FPS; the latter is an estimate since the output is now VFR (mpv will also show FPS fluctuating a bit if you open the output file in it and press I). I don't think the source video file matters, since I've been encountering this for a while now on varying source files.

I'm not sure if the problem also happens when selecting VP8 or VP9 from the options, but I couldn't test this since my build doesn't support those.

For what it's worth, I was also able to reproduce this against the latest mpv master that was built against the latest ffmpeg master (as of yesterday).

According to the mpv encoding docs, mpv seems to use VFR by default, so I suspect this issue is related to that. The docs also suggest using --vf=fps=RATE to enforce CFR, but I wasn't able to get this working when running a manual encoding command (output file is still VFR even if that flag is given, eg. mpv puipui.mp4 --start=0:01:18.161 --end=0:01:24.167 --ovc=libx264 --aid=no --vid=1 --sid=no --vf-add=fps=23.976 --ovcopts-add=crf=10 --o=test.mp4).

I'm unsure if the issue is really with mpv, and I'm kind of hesitant on raising an issue on the mpv repo about this, since I remember a maintainer saying that the encoding codebase isn't really maintained and may be removed at any time. I don't know if this is still the case.

Log file

mpv.log

ekisu commented 3 years ago

tl;dr: The video frame rate is being preserved, but the average frame rate reported by ffprobe is incorrect. It might be something in mpv, or ffmpeg, I'll try looking a bit more into it later, but it's definitively a bug upstream.


I've looked a bit into this, and it seems to be something specific with how frame stats are written into the MOV container. ffmpeg reads the average framerate of the track by looking into the stts atom (I'm not so sure on how things are called, but I think this is correct), and, by running ffprobe with tracing enabled, we can see that it has two entries (for some reason, a zero duration, one sample entry is added):

[mov,mp4,m4a,3gp,3g2,mj2 @ 000001a7a5ceef00] [trace] track[1].stts.entries = 2
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001a7a5ceef00] [trace] sample_count=479, sample_duration=1001
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001a7a5ceef00] [trace] sample_count=1, sample_duration=0

It interprets this as total_duration = sum(sample_count * sample_duration) and total_frames = sum(sample_count). The duration is in time_base units, and mpv always uses 1/24000 as the timebase, so we can figure out the calculated average frame rate with:

avg_frame_rate = total_frames / (time_base * total_duration)

Because we always have an extra zero duration, one sample entry (which adds an extra frame to the total_frames), the calculated average frame rate is always going to be slightly higher than the real one - longer videos will report an average frame rate closer to the real one. For this example, the formula gives ~24.026 FPS, which matches ffprobe's output:

  Stream #0:1[0x2](und): Video: h264 (High 10) (avc1 / 0x31637661), yuv420p10le(tv, bt709/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 267 kb/s, 24.03 fps, 23.98 tbr, 24k tbn (default)
    Metadata:
      creation_time   : 2021-03-29T21:46:28.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
sukerokushin commented 3 years ago

just ran into the same issue, thanks for the breakdown—eagerly hoping for a fix lol