search

joeyballentine / Video-Inference

Easy inference for video networks. Currently supports SOFVSR (traiNNer Version), RIFE, and TecoGAN-pytorch
53 stars 1 forks source link

Differing results between .mp4 and sequence upscaling #6

Open Sazoji opened 2 years ago

Sazoji commented 2 years ago

Weird blocking occurs when upscaling the same source video sequence (https://files.catbox.moe/3izib4.mp4) when rendered from png files vs rendering from extracted .mp4 frames, presumably with how ffmpeg-python grabs frames. Upscaled from .mp4 video (crf0): https://files.catbox.moe/545jh8.mp4 Upscaled from the same video extracted into a png sequence: https://files.catbox.moe/uu1bfv.mp4

Sazoji commented 2 years ago

Artifacts look like incorrect colorspace (rec 601 instead of 709, common with ffmpeg reading untagged SD video) unpresent in the source video and chroma issues (even when --denoise is enabled) which might be due to the nature of mpeg chroma subsampling.

Sazoji commented 2 years ago

Looking into this by seeing other projects with ffmpeg

joeyballentine commented 2 years ago

Been a bit busy and haven't looked into this. I guarantee it has something to do with that weird BGR -> RGB issue I was seeing and "fixed"

mirh commented 2 years ago

Colorspace problems are an awful reality with ffmpeg (and only the god knows what happens with the python wrapper) but at most you'd see a slight color shift, not "noisy glitches". EDIT: well, I'm seeing the same dirt even with standalone tacogan, so idk