Anime4Kcpp Video Mode

[mason1171]

Description

I see that the video2x.yaml contains this line: videoMode: true # Video process Despite setting the value as true, video2x still uses the frame extraction technique for Anime4Kcpp upscaling. This is much slower than the Anime4kcpp GUI program's video processing. Is this a bug or feature? Is it possible to skip the frame extraction and just upscale the video in question?

[k4yt3x]

Anime4KCPP's video processing doesn't give you much freedom. It will only use a fixed set of settings as supposed to the customizable settings in Video2X. As for speed, in the current mode of operation, Anime4KCPP just upscales all frames in a folder like how it's done with other drivers. This speed should theoretically be the same as native Anime4KCPP. If there's any difference in speed, it's most likely during the frame to video stage.

It is not possible to skip frame extraction. You'll understand why if you know how v2x works. Even if we switch to a frame server workflow, it is still impossible.

[mason1171]

Ok. Thanks. I guess the video mode option in the yaml has no effect then.

[k4yt3x]

Initially it did. When A4KCPP was initially added to v2x, the video just flows through Anim4KCPP's built-in pipeline in video mode. However, since its build-in pipeline isn't as flexible and customizable as v2x's FFmpeg pipeline, I moved that part of the work to be handled by v2x.

[TianZerL]

Native Anime4KCPP is faster because there is no frame extraction, frames data stream only flows in memory and video memory, and there is no extra overhead for compiling the OpenCL kernel. As k4yt3x said, the cost of doing so is less freedom.
BTW, the current speed bottleneck of Anime4KCPP GUI and CLI is encoding, that's why I'm using mp4v by default instead of avc1 in native Anime4KCPP.

Some suggestions for V2X:

• Process a folder containing extracted frames instead of individual frames, this will avoid extra overhead for compiling the OpenCL kernel. Because OpenCL kernel need to be compiled in runtime which means every time a new a4kcpp cli process is called, OpenCL kernel is compiled. You should process as many frames as possible in one a4kcpp cli process.
• If it is possible, do frames extraction in a ramdisk.
• Extracting all the frames at once requires a lot of disk space, consider to do it in sections.

[k4yt3x]

@TianZerL thanks for the suggestions.

• Process a folder containing extracted frames instead of individual frames, this will avoid extra overhead for compiling the OpenCL kernel. Because OpenCL kernel need to be compiled in runtime which means every time a new a4kcpp cli process is called, OpenCL kernel is compiled. You should process as many frames as possible in one a4kcpp cli process.

Right now the way it works is that all frames are extracted into one directory, and Anim4KCPP is run against that one directory. This is why I requested directory input/output in https://github.com/TianZerL/Anime4KCPP/issues/9.

• If it is possible, do frames extraction in a ramdisk.

Mounting a RAM disk would be a luxury for many, especially considering the size of the extracted frames for long videos. However, users do have the option to use a RAM disk if they have it mounted as a drive. They can then just point the temp directory to the RAM disk in the config file.

• Extracting all the frames at once requires a lot of disk space, consider to do it in sections.

I am aware of this, and I'm thinking whether I should do it in blocks or use a frame server, or both. It seems like processing the video in chunks might be more reasonable, at least so far.

[TianZerL]

@k4yt3x

Right now the way it works is that all frames are extracted into one directory, and Anim4KCPP is run against that one directory. This is why I requested directory input/output in TianZerL/Anime4KCPP#9.

I noticed that V2X created 16 empty folders named from 0 to 15 in processing, I guess V2X will assign all the frames to these 16 folders, but it doesn't seem to be the case. If V2X can do this and call a4kcpp clis in parallel to process the 16 folders, it will be faster.

[k4yt3x]

I just confirmed that to be true. Video2X will let the driver to handle multi-threading if the drive has native support for it. It's only when the driver doesn't support multi-threading natively will the folders be used. Multiple processes will then be created and run against each of the directories.