BabitMF / bmf

Cross-platform, customizable multimedia/video processing framework. With strong GPU acceleration, heterogeneous design, multi-language support, easy to use, multi-framework compatible and high performance, the framework is ideal for transcoding, AI inference, algorithm integration, live video streaming, and more.
https://babitmf.github.io/
Apache License 2.0
816 stars 71 forks source link

cpp copy module can't work with gpu transcoding #55

Open Jie-Fang opened 1 year ago

Jie-Fang commented 1 year ago

Module: test/c_module

def test():
    input_video_path = xxx
    output_path = xxxx
    video = bmf.graph().decode({
            "input_path": input_video_path,
            "video_params": {
                "hwaccel": "cuda",
            }
        })["video"]

    video2 = video.c_module('cpp_copy_module',
                            "../../test/c_module/libcopy_module.so", # use your path
                            "copy_module:CopyModule")

    (bmf.encode(
        video2,
        video["audio"],
        {
            "output_path": output_path,
            "video_params": {
                "codec": "h264_nvenc",
                "pix_fmt": "cuda",
            }
        }).run())

The output video isn't encoded normally. There're green and red area in the pictures.

But with CPU decoding and GPU encoding, the results are good.

def test():
    input_video_path = xxx
    output_path = xxxx
    video = bmf.graph().decode({
            "input_path": input_video_path,
        })["video"]

    video2 = video.c_module('cpp_copy_module',
                            "../../test/c_module/libcopy_module.so", # use your path
                            "copy_module:CopyModule")

    (bmf.encode(
        video2,
        video["audio"],
        {
            "output_path": output_path,
            "video_params": {
                "codec": "h264_nvenc",
            }
        }).run())
Jie-Fang commented 1 year ago

I found this issue when I tried to write a cpp module where the decoded NV12 frames will be converted to RGB, then the RGB frames are converted to NV12 again. The output video is similar as the results I describe above. So, I suspect there're some issues in the c_modules cooperating with gpu decoding.

sfeiwong commented 1 year ago

"tried to write a cpp module where the decoded NV12 frames will be converted to RGB, then the RGB frames are converted to NV12 again", so if you don't use this module but just copy_module, the issue still exist?