search

cea-sec / sanzu

Graphical remote desktop solution
GNU General Public License v3.0
182 stars 29 forks source link

[Windows] Server error due to set_option Undefined error -1414549496 #273

Closed JeremieSpiesser closed 7 months ago

JeremieSpiesser commented 7 months ago

Hello 👋, I'm trying to use sanzu_client on Linux (Ubuntu 22.04) to connect to a sanzu_server running on Windows (22H2). If I run the server with software encoding (CPU is an i5 6300u with an Intel HD Graphics 520) like so : sanzu_server.exe -l 0.0.0.0 -f sanzu.toml the client can connect just fine.

Now, if I run the server like so : sanzu_server.exe -l 0.0.0.0 -f sanzu.toml -e h264_qsv I get a weird error : 

[2024-02-26T15:56:16.558900300Z INFO  sanzu::server] Start server
[2024-02-26T15:56:19.589044100Z INFO  sanzu::server] Client 192.168.0.2:43694
[2024-02-26T15:56:19.590160300Z INFO  sanzu::server] Tls state: false
[2024-02-26T15:56:19.609636000Z INFO  sanzu::server] Client version Version { version: "0.1.1" }
[2024-02-26T15:56:19.626297700Z INFO  sanzu::server] Client screen size 3840x2160
[2024-02-26T15:56:19.627301900Z INFO  sanzu::server_windows] Create D3D11 device for 1
[2024-02-26T15:56:19.635568100Z INFO  sanzu::server_windows] Create window 1848 2057
[2024-02-26T15:56:19.647438200Z INFO  sanzu::server_windows] clipboard!
[2024-02-26T15:56:19.674507900Z INFO  sanzu::server_windows] D3D11 device ok for 1 0
[2024-02-26T15:56:19.675512500Z INFO  sanzu::server_windows] D3D11 0x2a7ba215718 45056 0x2a7ba21fb50
[2024-02-26T15:56:19.676623200Z INFO  sanzu::server_windows] known features level [45056, 41216, 40960, 37120]
[2024-02-26T15:56:19.695259200Z INFO  sanzu::server_windows] Query interface success 0x2a7ba214f28
[2024-02-26T15:56:19.697673900Z INFO  sanzu::server_windows] mode desc 1848x2057 refreshrate 66/1 format 87 scanlineorder 1 scaling 0
[2024-02-26T15:56:19.716978800Z INFO  sanzu::server_windows] rotation 1
[2024-02-26T15:56:19.718283700Z INFO  sanzu::server_windows] desktop in systemmem 0
[2024-02-26T15:56:19.730077400Z ERROR sanzu::server_windows] Change resolution: unsupported os
[2024-02-26T15:56:19.732326900Z ERROR sanzu_server] Server error
[2024-02-26T15:56:19.737074900Z ERROR sanzu_server]  - due to Error in init video encoder
[2024-02-26T15:56:19.751566400Z ERROR sanzu_server]  - due to Error set option error
[2024-02-26T15:56:19.752174300Z ERROR sanzu_server]  - due to set_option Undefined error -1414549496

My ffmpeg version seems to support h264_qsv

C:\Users\Test\Documents\sanzu-0.1.4-x86_64-pc-windows-gnu>ffmpeg -encoders
[...]
V..... h264_qsv             H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)
C:\Users\Test\Documents\sanzu-0.1.4-x86_64-pc-windows-gnu>ffmpeg -h encoder=h264_qsv
ffmpeg version 6.1.1-essentials_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --pkg-config=pkgconf --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-dxva2 --enable-d3d11va --enable-libvpl --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Encoder h264_qsv [H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration)]:
    General capabilities: delay hybrid
    Threading capabilities: none
    Supported hardware devices: qsv qsv qsv
    Supported pixel formats: nv12 qsv
h264_qsv encoder AVOptions:

I am using the sanzu-0.1.4 windows binaries from the release tab of this repo. Happy to provide more logs if needed to troubleshoot this.

serpilliere commented 7 months ago

Hi @JeremieSpiesser It seems to crash on a set_option of ffmpeg. Can you restart the process with de logger set to DEBUG instead of INFO? (using -vvv) This will display the current video option that sanzu sends to ffmpeg. Can you share the faulty option with us?

JeremieSpiesser commented 7 months ago

Thanks for the quick reply, here are the debug logs : 

[2024-02-27T14:20:56.955011600Z INFO  sanzu::server] Start server
[2024-02-27T14:21:11.333293800Z INFO  sanzu::server] Client 192.168.0.2:42554
[2024-02-27T14:21:11.334622700Z INFO  sanzu::server] Tls state: false
[2024-02-27T14:21:11.337775400Z INFO  sanzu::server] Client version Version { version: "0.1.1" }
[2024-02-27T14:21:11.350480900Z INFO  sanzu::server] Client screen size 3840x2160
[2024-02-27T14:21:11.351954300Z INFO  sanzu::server_windows] Create D3D11 device for 1
[2024-02-27T14:21:11.352183600Z INFO  sanzu::server_windows] Create window 1366 768[2024-02-27T14:21:11.356471300Z INFO  sanzu::server_windows] clipboard!
[2024-02-27T14:21:11.356774600Z INFO  sanzu::server_windows] Send clipboard r
[2024-02-27T14:21:11.356942200Z DEBUG sanzu::server_windows] Create window ok 0x400780
[2024-02-27T14:21:11.387168500Z INFO  sanzu::server_windows] D3D11 device ok for 1 0
[2024-02-27T14:21:11.387353800Z INFO  sanzu::server_windows] D3D11 0x1b391474458 45056 0x1b39147d710
[2024-02-27T14:21:11.388079500Z INFO  sanzu::server_windows] known features level [45056, 41216, 40960, 37120]
[2024-02-27T14:21:11.388505100Z INFO  sanzu::server_windows] Query interface success 0x1b391473c68
[2024-02-27T14:21:11.389568000Z INFO  sanzu::server_windows] mode desc 1366x768 refreshrate 76420000/1273600 format 87 scanlineorder 1 scaling 0
[2024-02-27T14:21:11.389781000Z INFO  sanzu::server_windows] rotation 1
[2024-02-27T14:21:11.389990400Z INFO  sanzu::server_windows] desktop in systemmem 0[2024-02-27T14:21:11.390567100Z ERROR sanzu::server_windows] Change resolution: unsupported os
[2024-02-27T14:21:11.391690000Z DEBUG sanzu::video_encoder] set_option: g -> 1500
[2024-02-27T14:21:11.392125700Z DEBUG sanzu::video_encoder] set_option: pixel_format -> yuv444p
[2024-02-27T14:21:11.392606000Z DEBUG sanzu::video_encoder] set_option: time_base -> 1/45
[2024-02-27T14:21:11.393059500Z DEBUG sanzu::video_encoder] set_option: b -> 3000000
[2024-02-27T14:21:11.393489600Z DEBUG sanzu::video_encoder] set_option: bf -> 0
[2024-02-27T14:21:11.393890900Z DEBUG sanzu::video_encoder] set_option: delay -> 0
[2024-02-27T14:21:11.394297200Z DEBUG sanzu::video_encoder] set_option: preset -> fast
[2024-02-27T14:21:11.394652000Z DEBUG sanzu::video_encoder] set_option: tune -> zerolatency
[2024-02-27T14:21:11.395774900Z ERROR sanzu_server] Server error
[2024-02-27T14:21:11.396456200Z ERROR sanzu_server]  - due to Error in init video encoder
[2024-02-27T14:21:11.397288900Z ERROR sanzu_server]  - due to Error set option error
[2024-02-27T14:21:11.402390000Z ERROR sanzu_server]  - due to set_option Undefined error -1414549496
serpilliere commented 7 months ago

Can you remove the zerolatency option in your sanzu.toml? and rerun it?

serpilliere commented 7 months ago

By the way, if you are using a h264_qsv decoder as well, don't forget to use a config.toml file on the client side. By default, some decoders makes "parallel" decoding (for example h264_qsv) but the draw back is that the decoding adds 4 frames in latency to the process... The config file for the client is very similar to the server side. For example:

[ffmpeg.h264_qsv]
pixel_format = "nv12"

async_depth = "1"

(this disable parallelisation, and then gain on decoding latency...

JeremieSpiesser commented 7 months ago 
[2024-02-27T14:29:57.650400600Z INFO  sanzu::server] Start server
[2024-02-27T14:30:00.016184300Z INFO  sanzu::server] Client 192.168.0.2:58062
[2024-02-27T14:30:00.016464800Z INFO  sanzu::server] Tls state: false
[2024-02-27T14:30:00.017215400Z INFO  sanzu::server] Client version Version { version: "0.1.1" }
[2024-02-27T14:30:00.025693100Z INFO  sanzu::server] Client screen size 3840x2160
[2024-02-27T14:30:00.026049300Z INFO  sanzu::server_windows] Create D3D11 device for 1
[2024-02-27T14:30:00.026203400Z INFO  sanzu::server_windows] Create window 1366 768
[2024-02-27T14:30:00.031545500Z INFO  sanzu::server_windows] clipboard!
[2024-02-27T14:30:00.032386800Z INFO  sanzu::server_windows] Send clipboard sanzu_server.exe -l 0.0.0.0 -e h264_qsv -f sanzu.toml
[2024-02-27T14:30:00.032710800Z DEBUG sanzu::server_windows] Create window ok 0x1b0454
[2024-02-27T14:30:00.056037900Z INFO  sanzu::server_windows] D3D11 device ok for 1 0
[2024-02-27T14:30:00.056414900Z INFO  sanzu::server_windows] D3D11 0x1f93da64718 45056 0x1f93da67b40
[2024-02-27T14:30:00.056876400Z INFO  sanzu::server_windows] known features level [45056, 41216, 40960, 37120]
[2024-02-27T14:30:00.057078700Z INFO  sanzu::server_windows] Query interface success 0x1f93da63f28
[2024-02-27T14:30:00.057899600Z INFO  sanzu::server_windows] mode desc 1366x768 refreshrate 76420000/1273600 format 87 scanlineorder 1 scaling 0
[2024-02-27T14:30:00.058131600Z INFO  sanzu::server_windows] rotation 1
[2024-02-27T14:30:00.058394000Z INFO  sanzu::server_windows] desktop in systemmem 0
[2024-02-27T14:30:00.058767600Z ERROR sanzu::server_windows] Change resolution: unsupported os
[2024-02-27T14:30:00.059888500Z DEBUG sanzu::video_encoder] set_option: g -> 1500
[2024-02-27T14:30:00.060135400Z DEBUG sanzu::video_encoder] set_option: bf -> 0
[2024-02-27T14:30:00.060335600Z DEBUG sanzu::video_encoder] set_option: pixel_format -> yuv444p
[2024-02-27T14:30:00.060599100Z DEBUG sanzu::video_encoder] set_option: time_base -> 1/45
[2024-02-27T14:30:00.060865000Z DEBUG sanzu::video_encoder] set_option: delay -> 0
[2024-02-27T14:30:00.061082200Z DEBUG sanzu::video_encoder] set_option: b -> 3000000
[2024-02-27T14:30:00.061254200Z DEBUG sanzu::video_encoder] set_option: preset -> fast
[2024-02-27T14:30:00.061649800Z ERROR sanzu_server] Server error
[2024-02-27T14:30:00.061870300Z ERROR sanzu_server]  - due to Error in init video encoder
[2024-02-27T14:30:00.062066600Z ERROR sanzu_server]  - due to Error in encoder open
[2024-02-27T14:30:00.062265800Z ERROR sanzu_server]  - due to avcodec_open2 Undefined error -22

My config for the server looks like this : 

[video]
max_fps = 60
max_stall_img = 35
control_path = "/var/run/surf_video_control/client_%PID%"

[audio]
sample_rate = 44100
max_buffer_ms = 200

[ffmpeg.global]
b = "3000000"
time_base = "1/45"
delay = "0"
# gop_size (25 * 60 = 1500)
g = "1500"
# max_b_frames
bf = "0"
#pixel_format = "yuv420p"
pixel_format = "yuv444p"

[ffmpeg.libx264]
preset = "fast"
tune = "zerolatency"

[ffmpeg.h264_qsv]
preset = "fast"

[ffmpeg.libx265]
preset = "fast"
tune = "zerolatency"

I'm not using an hardware accelerated decoder (yet) but I will remember to configure it like you said.

serpilliere commented 7 months ago

I think the h264_qsv does not support yuv444p. Can you try with yuv420p?

By the way, you can put the pixel format in the 'codec' part instead of global. Like that, the pixel format will depend on the codec using, this will avoid modifying the config file on each codec test

JeremieSpiesser commented 7 months ago

Same error apparently : 

[2024-02-27T14:35:49.528678900Z INFO  sanzu::server] Start server
[2024-02-27T14:35:52.153442500Z INFO  sanzu::server] Client 192.168.0.2:37292
[2024-02-27T14:35:52.153702300Z INFO  sanzu::server] Tls state: false
[2024-02-27T14:35:52.154356700Z INFO  sanzu::server] Client version Version { version: "0.1.1" }
[2024-02-27T14:35:52.162633100Z INFO  sanzu::server] Client screen size 3840x2160
[2024-02-27T14:35:52.162942500Z INFO  sanzu::server_windows] Create D3D11 device for 1
[2024-02-27T14:35:52.163114500Z INFO  sanzu::server_windows] Create window 1366 768
[2024-02-27T14:35:52.167284000Z INFO  sanzu::server_windows] clipboard!
[2024-02-27T14:35:52.167606700Z INFO  sanzu::server_windows] Send clipboard pixel_format = "yuv420p"
[2024-02-27T14:35:52.167843400Z DEBUG sanzu::server_windows] Create window ok 0x140292
[2024-02-27T14:35:52.190841200Z INFO  sanzu::server_windows] D3D11 device ok for 1 0
[2024-02-27T14:35:52.191097900Z INFO  sanzu::server_windows] D3D11 0x1c0cbcd1788 45056 0x1c0cbcdaa40
[2024-02-27T14:35:52.191572700Z INFO  sanzu::server_windows] known features level [45056, 41216, 40960, 37120]
[2024-02-27T14:35:52.191819700Z INFO  sanzu::server_windows] Query interface success 0x1c0cbcd0f98
[2024-02-27T14:35:52.192681800Z INFO  sanzu::server_windows] mode desc 1366x768 refreshrate 76420000/1273600 format 87 scanlineorder 1 scaling 0
[2024-02-27T14:35:52.192903100Z INFO  sanzu::server_windows] rotation 1
[2024-02-27T14:35:52.193096300Z INFO  sanzu::server_windows] desktop in systemmem 0
[2024-02-27T14:35:52.193503400Z ERROR sanzu::server_windows] Change resolution: unsupported os
[2024-02-27T14:35:52.194665200Z DEBUG sanzu::video_encoder] set_option: time_base -> 1/45
[2024-02-27T14:35:52.194900900Z DEBUG sanzu::video_encoder] set_option: delay -> 0
[2024-02-27T14:35:52.195122900Z DEBUG sanzu::video_encoder] set_option: bf -> 0
[2024-02-27T14:35:52.195285500Z DEBUG sanzu::video_encoder] set_option: b -> 3000000
[2024-02-27T14:35:52.195510700Z DEBUG sanzu::video_encoder] set_option: pixel_format -> yuv444p
[2024-02-27T14:35:52.195772500Z DEBUG sanzu::video_encoder] set_option: g -> 1500
[2024-02-27T14:35:52.196007400Z DEBUG sanzu::video_encoder] set_option: preset -> fast
[2024-02-27T14:35:52.196201300Z DEBUG sanzu::video_encoder] set_option: pixel_format -> yuv420p
[2024-02-27T14:35:52.196654600Z ERROR sanzu_server] Server error
[2024-02-27T14:35:52.196867400Z ERROR sanzu_server]  - due to Error in init video encoder
[2024-02-27T14:35:52.197032300Z ERROR sanzu_server]  - due to Error in encoder open
[2024-02-27T14:35:52.197214800Z ERROR sanzu_server]  - due to avcodec_open2 Undefined error -22
serpilliere commented 7 months ago

Hum.. Which ffmpeg do you use? Are you sure the dll is compiled with the h264_qsv support ?

JeremieSpiesser commented 7 months ago

I followed these instructions : https://github.com/cea-sec/sanzu/tree/main/sanzu#launching-binaries-on-windows. Therefore, my ffmpeg comes from https://github.com/BtbN/FFmpeg-Builds/releases. Is there a way to check that the dll is compiled with h264_qsv support ?

JeremieSpiesser commented 7 months ago 
C:\Users\Test\Documents\ffmpeg-master-latest-win64-gpl-shared\bin>ffmpeg.exe -encoders
ffmpeg version N-113699-g0c8e64e268-20240219 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 13.2.0 (crosstool-NG 1.25.0.232_c175b21)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --enable-shared --disable-static --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libharfbuzz --enable-libvorbis --enable-opencl --disable-libpulse --enable-libvmaf --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libaribcaption --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20240219

According to this SO post https://superuser.com/questions/1428442/using-ffmpeg-to-encode-a-video-to-h264-using-intel-quicksync I would need --enable-libmfx to be there (which it is not) to have h264_qsv is that right ?

In either case, encoding a locally saved video works with h264_qsv 

C:\Users\Test\Documents\ffmpeg-master-latest-win64-gpl-shared\bin>ffmpeg.exe -i input.webm -vcodec h264_qsv -crf 0 output.mp4

and stresses the Intel GPU

serpilliere commented 7 months ago

Is there a way to check that the dll is compiled with h264_qsv support ?

There is an API that we may call in Sanzu to list encoders (to mimic the --encoders of ffmpeg binary). I will add this feature to the current version of Sanzu: I think this is an interesting one.

serpilliere commented 7 months ago

In either case, encoding a locally saved video works with h264_qsv

Your test & conclusion are perfects:

If I am correct, the pixel format supported by the h264_qsv encoder is nv12 and not yuv420p (sorry for that) You can see it with the command:

ffmpeg  -h encoder=h264_qsv
ffmpeg version n6.1 Copyright (c) 2000-2023 the FFmpeg developers
...
    Supported pixel formats: nv12 qsv

(This format is supported by Sanzu)

Which differs from the libx264 for example:

ffmpeg  -h encoder=libx264 
ffmpeg version n6.1 Copyright (c) 2000-2023 the FFmpeg developers
...
    Supported pixel formats: yuv420p yuvj420p yuv422p yuvj422p yuv444p yuvj444p nv12 nv16 nv21 yuv420p10le yuv422p10le yuv444p10le nv20le gray gray10le

So can you retry with

pixel_format="nv12"

instead of yuv420p?

By the way, this was the default config in the example file (sanzu.toml):

...
[ffmpeg.h264_qsv]
pixel_format = "nv12"

look_ahead = "0"
# No parallelism => no latency
async_depth = "1"
...
JeremieSpiesser commented 7 months ago

Okay that was it (pixel_format="nv12") 🎉. Thanks for your help. I wrongly extrapolated the demo/server/sanzu.toml instead of seeing that there was a file in sanzu/sanzu.toml 🤦