JossCamp
commented
1 week ago VideoJaNai uses vapoursynth to process videos, I imagine it must be a problem in the processing pipeline between the model and the frame processing, I thought I was the only one who seemed to have that problem.
fideliochan
I've converted ESRGAN's AnimeVideoV3 to ONNX to use in VideoJaNai, however the output video is unfaithful compared to using the Windows build that uses NCNN aswell as the converted ONNX model if used in ChaiNNer. VideoJaNai makes the models lose some detail that ChaiNNer would have otherwise kept. I've tried everything I can think of. Switching from TensorRT to NCNN, Altering the TensorRT Engine parameters, choosing not to resize the video before or after upscaling, resizing the video manually beforehand with ffmpeg, encoding losslessly and using FP16 converted model instead of FP32.
Regular ESRGAN:
VideoJaNai:
Notice how at the top left, the black lines in the hair is blurred away in the VideoJaNai version. The very little that was left of the nose almost vanished with VideoJaNai. Even the included 2x_AnimeJaNai_HD_V3_Compact.onnx behaves differently in chaiNNer compared to VideoJaNai.
ChaiNNer:
VideoJaNai:
Some weird colors using that in ChaiNNer for some reason, but either way it has the same issue, there is detail loss when using VideoJaNai, regardless of what setting I change in it.