ffmpeg 4.4 breaks NVENC on older GPUs

[guihkx]

Hello!

I own a GTX 660 and since the ffmpeg 4.4 update (Flathub commit c626cb7538d96e5ffa037d1d23b004e2cffca6cf8a9b903741e5664a7a4f3829), I'm not able to record using NVENC because most of the frames (80%+) get skipped by OBS.

Downgrading OBS to commit 6147068726939e0551e004ddd8f8c524e45969f86d9749891018b32bd0437d90 fixes the issue.

I've reported this issue upstream w/ a few more details: https://github.com/obsproject/obs-studio/issues/4631

[GeorgesStavracas]

@guihkx would you be so kind to test the build at https://github.com/flathub/com.obsproject.Studio/pull/132, and let me know if it actually fixes the regression with ffmpeg 4.4?

[guihkx]

Working beautifully! Here's the result of a 10 second recording:

info: ---------------------------------
info: [NVENC encoder: 'recording_h264'] settings:
    rate_control: CBR
    bitrate:      2500
    cqp:          0
    keyint:       250
    preset:       hq
    profile:      high
    width:        1920
    height:       1080
    2-pass:       false
    b-frames:     2
    GPU:          0

info: libfdk_aac encoder created
info: libfdk_aac bitrate: 160, channels: 2
info: ==== Recording Start ===============================================
info: [ffmpeg muxer: 'adv_file_output'] Writing file '/home/gui/2021-05-13 18-57-46.mkv'...
info: Output format name and long_name: matroska, Matroska
info: [ffmpeg muxer: 'adv_file_output'] Output of file '/home/gui/2021-05-13 18-57-46.mkv' stopped
info: Output 'adv_file_output': stopping
info: Output 'adv_file_output': Total frames output: 674
info: Output 'adv_file_output': Total drawn frames: 686
info: ==== Recording Stop ================================================
info: libfdk_aac encoder destroyed

However, I must note that initially I was getting this error:

info: [NVENC encoder: 'recording_h264'] settings:
    rate_control: CBR
    bitrate:      2500
    cqp:          0
    keyint:       250
    preset:       hq
    profile:      high
    width:        1920
    height:       1080
    2-pass:       false
    b-frames:     2
    GPU:          0

[h264_nvenc @ 0x55add06b1980] dl_fn->cuda_dl->cuInit(0) failed -> CUDA_ERROR_UNKNOWN: unknown error
warning: [NVENC encoder: 'recording_h264'] Failed to open NVENC codec: Unknown error occurred

I tried to reboot, but the message didn't go away. Only after a few times restarting OBS, it started working again.

Could have been just the NVIDIA driver acting weird, idk.