search

master-of-zen / Av1an

Cross-platform command-line AV1 / VP9 / HEVC / H264 encoding framework with per scene quality encoding
GNU General Public License v3.0
1.51k stars 156 forks source link

Question about VFR encoding #828

Open ioctl-user opened 7 months ago

ioctl-user commented 7 months ago

I have 4k 60 FPS video from the mobile phone.

Actually, it can be classified as VFR, because it seems there is dropped about one frame every two seconds.

I'm trying encode it with scaling with the following command:

av1an -i in.mp4 -m lsmash --extra-split-sec 2 -e  svt-av1 -v "--preset 12" -f " -vf scale=1920:-1:flags=lanczos "  --vmaf-filter "scale=1920:-1:flags=lanczos" --target-quality 97  -o output.mp4

Output video has the same frame number as an input, according to the ffprobe. However, VMAF value is about 18.5 and VMAF chart looks very suspiciously.

output

Looks like the first 95 frames has good synchronization and then something goes wrong.

How can I fix this in av1an?

Using ffmpeg the same video can be encoded and VMAF correctly measured with the following commands:

ffmpeg -y -vsync vfr -i in.mp4 -vf scale=1920:-1:flags=lanczos -c:v libsvtav1 -tune 2 -preset 12 -crf 35 -svtav1-params enable-qm=1:qm-min=0 out.mp4

ffmpeg -i out.mp4 -i in.mp4 -filter_complex '[0:v]fps=fps=60,setpts=PTS-STARTPTS[distorted]; [1:v]scale=1920:-1:flags=lanczos,fps=fps=60,setpts=PTS-STARTPTS[reference]; [distorted][reference]libvmaf=model=path=/usr/share/model/vmaf_v0.6.1.json:feature=name=psnr:log_path=out.log:log_fmt=xml:n_threads=12' -f null -
master-of-zen commented 6 months ago

Hmmmm, from what i remember in av1an we actually redo the frame time so that it's always next frame to next frame from both streams regardless the dts match. However, that doesn't negate situation when frame order/count changes

ioctl-user commented 6 months ago

I just did more investigations. Actually, it seems, the only problem in input video is somewhere dropped frames. There was no other frames timestamp irregularity.

ioctl-user commented 6 months ago

Here is an reproducible example with the generated video.

So, generate video with each 40-th frame is dropped:

ffmpeg -f lavfi -i mandelbrot=end_pts=50:size=640x480 -pix_fmt yuv420p10le -vf "select='mod(n,40)'" -t 10 mandelbrot-in-drop.mkv

Encode it:

av1an -i mandelbrot-in-drop.mkv -m lsmash --extra-split-sec 2 -e  svt-av1 -v "--preset 12" --target-quality 97  -o mandelbrot-out.mkv

VMAF picture: mandelbrot-out