search

n00mkrad / flowframes

Flowframes Windows GUI for video interpolation using DAIN (NCNN) or RIFE (CUDA/NCNN)
GNU General Public License v3.0
1.52k stars 117 forks source link

Severe banding issues #167

Closed GlasslessPizza closed 2 years ago

GlasslessPizza commented 2 years ago

After an afternoon of testing, I finally was able to determine why my outputs had color banding issues that weren't present in the original footage. In short, FFmpeg essentially defaults to some generic limited-range output for extracted frames in the PNG format. To solve the situation it is necessary to force full-range output by inserting the following parameters in the command line of the extraction step in Flowframes: -sws_flags +accurate_rnd+full_chroma_int -pix_fmt rgb24 or for HDR content: -sws_flags +accurate_rnd+full_chroma_int -pix_fmt rgb48be However the format "rgb48be" blows up space utilization 4x and I can't think of a workaround short of implementing streamed extraction/chunking.

I've been able to workaround this by separately extracting png frames using the above parameters and then feeding the folder to Flowframes. A word of advice, for anybody that still experiences banding issues or color depth issues even after doing this, don't be afraid to tweak the Quality Level under Encoding Settings and changing to yuv420p10le for the interpolated video.

The "Import HQ JPEGs instead of PNGs" option is only slightly less affected, but enabling that introduces other discoloration issues which is a different can of worms.

GlasslessPizza commented 2 years ago

After further investigation, it turns out the solution i posted above isn't really a general one and will only help for very specific files. I tried so many combinations of parameters to get ffmpeg to emit decent PNGs but I just couldn't. This is surely very bad luck when it comes to the parameter combinations i tried. The one successful combo that halfway consistently gets rid of banding artifacts is the following extremely specific one that uses the new jpegxl:

ffmpeg -i input.mkv -distance 0.5 -effort 5 -sws_flags +accurate_rnd+full_chroma_int -pix_fmt rgb48be "extracted/almostlossless_%%03d.jxl"

The problem is that jpegxl is too slow to be useful and Flowframes doesn't support it anyway. I'll ask around and come back if i find anything helpful.

n00mkrad commented 2 years ago

Probably best to just use RIFE-NCNN-VS, can you provide a sample of the problematic video?

GlasslessPizza commented 2 years ago

Ok I attach here the input video that demonstrates the problem and the output after i run it through the CUDA/pytorch backend. I suggest using VLC to view the videos for comparison because every other video player i tried won't show it accurately. RIFE-NCNN-VS indeed isn't affected as it bypasses that step entirely. I have a different problem with that video using RIFE-NCNN-VS but i'll open another issue as it's offtopic here. samples.zip

GlasslessPizza commented 2 years ago

Ok I realized what the problem was. In short, I have been viewing the videos for comparison in VLC using the Always Fit Window option. This was a bad idea because I didn't know that VLC manipulates aggressively the video if it's not played at native resolution. It seems that one of these manipulations involves removing banding artifacts that were actually present in the original video. FFmpeg and Flowframes are not to blame for this. I'll close this issue for now unless I notice something else.