ffmpeg h264_nvenc options aren't lowest latency possible - feature request to customize encoder options

[digitalsignalperson]

I noticed that I could get almost imperceptible latency streaming with just ffmpeg and netcat, so I looked closer at what settings Weylus is using and why I get noticeable lag.

Here's a local test to use x11grab and stream it locally to mpv, using settings with imperceptible latency

ffmpeg -video_size 2560x1440 -framerate 25 -f x11grab -i :0.0+2560,1080 -c:v h264_nvenc -preset:v llhq -tune ull -zerolatency 1 -delay 0 -f mpegts - | mpv  --profile=low-latency --untimed --no-demuxer-thread -

and then switching to the settings weylus is using https://github.com/H-M-H/Weylus/blob/6411ba45260859f6ba44eb56ba86af4d5227eab3/lib/encode_video.c#LL313C45-L313C45

ffmpeg -video_size 2560x1440 -framerate 25 -f x11grab -i :0.0+2560,1080 -c:v h264_nvenc -preset:v fast -tune ull -zerolatency 1 -rc vbr -cq 21 -f mpegts - | mpv  --profile=low-latency --untimed --no-demuxer-thread -

It's pretty fast, but just hitting the Super key to pop open my application menu it's easy to see the delay from keypress to the response. Versus it's instant with the other settings.

So the best settings I have:

• preset = llhq
• tune = ull
• zerolatency = 1
• delay = 0

And what weylus uses:

• preset = fast
• tune = ull
• zerolatency = 1
• rc = vbr
• cq = 21

Changing my own setting from preset = llhq to fast is enough to add what seems like the delay I see with weylus. But changing weylus from fast to llhq still has a delay. I think weylus needs at least preset = llhq and delay = 0

But maybe weylus should just expose these arguments and let the user tweak them to what works best in their setup.

[digitalsignalperson]

trying it out here https://github.com/digitalsignalperson/Weylus/tree/llhq

I realized I'm getting low latency on the default h264 and that is being defaulted to when nvenc can't encode resolutions above 4096 giving InitializeEncoder failed: invalid param (8): Invalid Level.

Yet at lower resolutions, nvenc through weylus is very slow compared to the ffmpeg/mpv/netcat setup which I can't figure out why

[TheDogHusky]

I have a related problem: I used nvenc and the refresh rate of the screen on my iPad mini 4 was like 30s for each frames and it was completely bugged, like the cursor wasn't updating, the screen was updating only when I changed window or tab in brave, which is really weird. I disabled acceleration in weylus and the issue is fixed, but now with kind of low fps.